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To the President and the Congress of the United States; 

Chi behalf of the National Commission for Employment Policy, I am pleased to submit this report 
of findings and recommendations on using Unemployment Insurance (UT) wage-record data for 
performance management of programs funded under the Job Training Partnership Act (JTPA). The 
report deals with performance standards for economically disadvantaged adults who have completed 
JTPA's Title H-A programs and for dislocated workers who have completed JTPA's Title 
WjEDWAA programs. 

When JTPA was enacted in T982, it was the fust tinK that a rruijor domestic social program had a 
strong "outcomes" orientation. The President and Congress viewed JTPA's training programs for 
economically disadvantaged youth and adults and for dislocated workers as an investment, one which 
was to reap a positive return. The Act established a strong performance management system, with 
performance standards - measures of the return on investment -- as a central component. 

Over time the Department of Labor has been refining these performance standards and the 
methods of collecting the outcomes data necessary to implement them. In the early years of the 
program, the Department required data on such performance measures as "employment at time of 
termination from the program * and "wage at placement" for adults. 

As the system matured, the Department shifted its emphasis to longer-term measures of program 
performance; this shin reflected the Department's on-going concern that the program have a 
long-term impact on its participants' employment and earnings. By 1986 the Department required 
the collection of follow-up data. 

While this shift in emphasis was an important step forward in assessing the effect of the training 
programs, it has had significant adverse implications for program administration nd administrative 
costs. To obtain the required information, program terminees are surveyed by telephone during the 
thirteenth week after they leave the program. Implementation is difficult and costly. 

Since this survey method was instituted, an alternative way to collect employment and earnings 
information has emerged. As a result of the Deficit Reduction Act of 1984, effective September 
1988, the employers in each State are required to submit quarterly wage reports to a State agency 
(which may be the agency administering the State's unemployment compensation law). At present, 



all but two States meet this mandate through their UI wage reporting system. These now widely 
available UI wage data offer a low-cost way to obtain the needed basic information about JTPA's 
performance. 

This report addresses the technical and administrative issues involved in taking advantage of this 
new data source. It also shows the usefulness of the UI wage data in measuring ihe long-term 
effectiveness of JTPA's services and documents cost savings in excess of $2 million annually. 
Finally, the report suggests the potential value of UI wage-record data as a "common currency" for 
measuring the performance of JTPA and other employment and training programs. 

This report underpins the Commission's recommendation to provide States with an option to use 
the UI wage-record data, in place of the telephone survey data, for JTPA performance standards. 
While a period of transition, and specific technical assistance on the part of the Department of Labor, 
will be required to implement this option, the long-term benefits to the JTPA system are great. 

Now more than ever, the Nation needs to know how well its training programs are doing because 
the training needs of America s workforce are growing. At the same time, government budgets are 
tightly constrained; every dollar is stretched to its limit. The Commission is recommending a more 
cost-effective way of measuring JTPA's return on investment 

JohnC. Gartland 
Chairman 
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■ SUMMARY AND 
RECOMMENDATIONS 



Introduction 



The National Com* nission for Employment Polity (NCEP) has sponsored a project to investigate 
the potential for States to use Unemployment Insurance (UI) wage records to track the 
employment and earnings experiences of participants in programs provided under the Job 
Training Partnership Act (JTPA). This part of the report presents a summary of the project and 
the Commission's recommendations. The next part presents the studies that formed the basis 
for these recommendations. 

This introductory section contains the ■ tionale for using the UI wage-record data for JTPA's 
performance standards/ the central component of its system of performance management The 
second section pi ovides a brief background on the studies that have comprised the project The 
third section contains the recommendations adopted by the Commission at its 64th Meeting on 
September 19-20, 1991. 

The UI wage records are a promising alternative to the telephone survey now used by the States 
to measure post-program employment and earnings in order to implement federal performance 
standards for JTPA programs. These standards are driven by the legislative provision that job 
training is an investment in human capital and that the basic return on this investment shall be 
measured in terms of the increased employment and earnings of participants and the reductions 
in welfare dependency. 1 

The wage records contain the quarterly earnings of workers employed in firms covered by 
States' UI systems. States maintain the records in order to administer unemployment insurance 
benefits and taxes. This NCEP project has examined the possibility that these UI data can also 
be used to track the labor market experiences of JTPA participants. 

The UI wage records have great potential as a data resource for JTPA performance management 
and program evaluation because they offer the following benefits: 

• The ability to obtain vital information about the employment and earnings of virtually 
all JTPA participants at all levels of service delivery. 

• The ability to link the employment and earnings of JTPA participants to a number of 
important factors, including participants' characteristics (e.g., age and education) and 
types of program services (e.g., occupational classroom training and on-the-job 
training). 

• The ability to track the employment and earnings of participants by quarter over long 
periods of time before and after receiving employment and training services. 
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♦ The potential ability to evaluate the net impact of program services through the use of 
a comparison group for which employment and earnings can be measured in the same 
way as for the participant group. 

Moreover, the use of UI wage-record data in place of the telephone survey data, now used for 
JTPA performance standards, promises large annual savings in the costs of data collection for 
the States. For example, be sed on information provided by nineteen States participating in the 
NCEP Project, the estimated combined savings for the Title II-A adult and Title ffl/EDWAA 
programs ranged from $4,500 per year for a relatively small State to $300,000 per year for a very 
large State. 2 These savings are the difference between the annual costs of the telephone survey 
in Program Year 1990 and the estimated operating costs of using the UI wage records had they 
been used instead of the survey. 

Such annual savings in the costs of data collection may be offset somewhat if States incu* 
increased costs of data analysis. This could occur if States deride to use the UI data to evaluate 
JTPA outcomes more closely man is possible with the survey data. However, the States are likely 
to reap large cost savings in any event. 

The National Governors' Association (NG A) estimated similar cost savings: the average cost 
for follow-up is about $2 per JTPA participant using the UI wage records, while the average cost 
is about $19 per completed interview using the current telephone survey. The total annual 
cost to the States of collecting the follow-up information required for performance standards 
under the JTPA Title II-A (adult) and Title III/EDWAA programs would be about $2 million 
less if the UI wage records were used rather than the survey. 

NG A also found that start-up costs for the State UI offices ranged from $600 in a small State to 
$20,000 in a large State. Generally, the annual savings to the States that can be obtained by using 
the UI wage records rather than the survey are much larger than these start-up costs. 

Project Background 



This NCEP Project began in 1988 when eleven States Joined the NCEP in a feasibility study of 
linking records of the JTPA participants with wage records from the States' UI systems. The 
State of Illinois, with Northern Illinois University as its agent, served as a clearinghouse for 
integrating the data from the States. The eleven States provided UI wage data for their JTPA 
participants who terminated from the program during Program Year 1986. 

The NCEP issued a Research Report on these initial investigations, A Feasibility Study of the 
Use of Unemployment Insurance Wage-Record Data as an Evaluation Tool for 1TPA: Report on 
Project's Pha«e I Activities (January 1991). Because of the encouraging nature of the report's 
findings, in conjunction with growing interest by States in using the UI data for performance 
management, the Commission expanded the project in two phases. In Phase II, four new States 
and data for Program Year 1987 were added; in Phase III, five more States and data for Program 
Year 1988 were added. The US. Department of Labor (DOL) provided some financial support 
for this expansion. 

The project also included the three studies presented in Chapters II to IV below- A wide range 
of technical and administrative issues involved in using the UI data for JTPA performance 
management were addressed. 4 The studies were. 

• A Canvass of All States conducted by the National Governors' Association. This 
study obtained information on the barriers to data-sharing agreements between the 
JTPA and UI agencies, staff resource and cost constraints that impede the linkage of 




the two data sets, and methods that States have developed to overcome these barriers 
and constraints. 

• A Confidentiality Study conducted by Dr. David Stevens of the University of 
Baltimore. This study analyzed the key confidentiality provisions found in State 
statutes and accompanying administrative documents. This information was 
obtained from the State Employment Security Agency administrators of all States. 

• A Technical Feasibility Study conducted by Northern Illinois University. This study 
used the linked data to examine response biases in the survey data, coverage problems 
with the UI data, and the dynamics of the labor market experiences of JTPA 
participants after they leave the program. 

The findings from these studies indicate that it is feasible for many States to use the UI 
wage-record data for JTPA performance standards, provided that a transitional approach is 
taken. These findings were discussed at an NCEP Project Workshop held in Alexandria, Virginia 
on September 5-6, 1991. The workshop served as a forum for the State representatives to offer 
their views on the feasibility and desirability of using the UI wage records for JTPA performance 
standards. Also attending the research presentations were staff from the VS. Department of 
Labor, National Occupational Information Coordinating Committee, and Interstate Conference 
of Employment Security Agencies. 

More than half of the representatives present at the workshop predicted that their States would 
adopt the UI approach in Program Year 1992 if given the choice. The other representatives 
predicted that their States would follow soon thereafter, probably in Program Year 1993. 

The next section presents the recommendations adopted by the National Commission for 
Employment Policy at its 64th meeting on September 19-20, 1991. These recommendations are 
aimed at facilitating the earliest effective use of the UI wage records for JTPA performance 
management 

Recommendations 



These recommendations would give the VS. Department of Labor (DOL) a transitional year in 
which to adapt its national methodology for performance standards to use of the UI wage data. 
In addition, these recommendations envision that each State would require at least one year to 
adapt its State procedure for implementing performance standards to use of the UI data. For 
example, the State of Illinois has already begun this transition. Other States could choose to start 
in Program Year 1992. 

RECOMMENDATION 1: The Department of Labor should offer States the 
opportunity in Program Year 1992 to conduct pilot projects in which the UI 
wage-record data are used in the States' implementation of performance 
standards and associated incentive grants. These projects could involve either 
individual States or regional consortia of States, depending on the States' 
desire to pool data and other resources. 

The participating States would employ both the UI wage-record data and the telephone survey 
data for direct comparison during the application of performance standards. 6 The NCEP is 
recommending that the DOL fund the start-up and operating costs of accessing the UI data for 
all States evidencing the readinese to conduct the pilot projects. The States would remain 
responsible for the costs of the survey data during this transitional year. As indicated earlier, 
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the start-up costs of accessing the UI data arc generally small relative to the resultant annual 
savings in data collection costs. 

Since the survey data would be collected for all States *n Program Year 1992, it would be possible 
for DOL to maintain its current national methodology during the first half of the next two-year 
cycle of performance standards. 

This transitional approach would give DOL a year in which to test alternative performance 
measures and standards adjustment models based on the UI data. NCEP has shared the Ul data 
collected through its project with DOL, so that these tests are under way. As part of its 
development agenda, DOL could compare adjustment models estimated from individual data 
for JTPA participants to model* estimated from aggregate data for Service Delivery Areas 
(SDAs). 

The proposed use of individual data has considerable appeal, since state-specific models can be 
developed from the individual data but not the SDA-aggregate data. 7 State-specific models 
based on individual UI data might provide a more equitable basis for adjustments in SDA 
performance standards than would a national model based on either individual or 
SDA-aggregate UI data. 

As another part of its development agenda, DOL could examine the possibility of restoring 
immediate outcomes (e.g., wage rate at placement) to the national set of performance measures 
by Program Year 1994. The UI wage-record data become available one to two quarters later than 
the survey data. This delay raises issues about the timeliness of the UI data for (1) federal 
oversight of the JTPA system, (2) State policy concerning incentive grants and sanctions, and (3) 
local program management. The restoration of immediate outcomes to the set of measures for 
performance standards, to be used in close conjunction with the follow-up measures, would 
respond to these issues of timing. 

In addition, DOL would have a transitional year in which lo develop the technical assistance 
package needed by many States to use the UI data for performance standards. This need is taken 
up in Recommendation 5 below. 

Regional consortia may be necessary for two reasons. First, for some SDAs, out-of-state 
employment is extensive (e.g., the Cincinnati, St. Louis and Washington, D.C. areas). In these 
cases, States may not be satisfied with a statistical adjustment procedure, as envisioned in 
Recommendation 4 below. Interstate data sharing may be the preferred way to assure equity 
in the application of performance standards among the SDAs. 

Second, some States (small ones in particular) do not currently have the staff resources to take 
full advantage of the UI data for purposes of performance management and related program 
evaluation. Multi-state consortia may be the cost-effective way for these States to process and 
analyze the UI data. 

RECOMMENDATION 2: States should be offered the choice starting in 
Program Year 1993 to use the Ul wage-record data in place of the telephone 
survey data for JTPA performance standards. States would be able to 
discontinue the survey upon demonstrating the capacity during a transitional 
year to use the Ul data in implementing all aspects of performance standards 
and associated incentive grants. 

During their transitional year the States would collect both the Ul wage-record data and the 
telephone survey data for direct comparison during the States' application of performance 
standards. The NCEP is recommending th a the DOL fund the start-up and operating costs of 



accessing the Ul data for all States choosing to make the transition. The States would remain 
responsible for the costs of the survey data during their transitional year. 

States that conduct pilot projects in Program Year 1992 would have accomplished their 
transition. They would be able to discontinue the telephone survey in Program Year 1993 and 
thereby reap the cost savings associated w* h the UI data. The other Stales would have this 
opportunity as early as Program Year 1994, depending on when they choose to make their 
transition. States would be responsible for the costs of accessing the UI data upon 
discontinuation of the survey. 

In addition to the experience obtained from the pilot projects in Program Year 1992, the States 
exercising the choice in Recommendation 2 would be able to draw upon the technical assistance 
developed by DOL during that >«;*r. States could choose to use the Ul data through formal 
statements of intent and capacity in their State plans submitted to DOL. 

A choice-based system, such as this, raises complications for DOL with regard *o the federal 
reporting and communication of JTPA outcomes. The NCEP is sensitive to these complications 
and would work with DOL to avoid the emergence of a "patchwork quilt" of performance 
standards. Universal adoption of the UI data may be the long-run solution. 

As the States and DOL gain greater experience in using the UI data for performance standards, 
universal adoption of the UI data may become feasible, perhaps as early as Program Year 1996. 
Universal adoption is desirable on many grounds. For example, post-program earnings would 
then be measured in the same way by all States. 9 This uniformity would facilitate federal 
reporting and communication of JTPA results to policymakers and the public. It also would 
enhance coordination between the JTPA program and other employment and training programs 
(e.g., those under Vocational Education, AFDC/JOBS, and Food Stamps) by fostering a 
"common currency" for performance standards. 

In support of overarching Recommendations 1 and 2, the NCEP also offers the following 
recommendations. 

RECOMMENDATION 3: States that choose to use the Ul wage-record data 
for performance management should develop written agreements for all 
resultant UI data-sharing activities. 

These written agreements should assure responsible and prudent management of the UI data. 
Such agreements would be necessary when States wished to share Ul data with each other to 
obtain information on out-of-state employment or with their SDAs to aid local program 
management. 

RECOMMENDATION 4; The Department of Labor should develop a 
procedure to make statistical adjustments in JTPA performance standards for 
variations in UI coverage and out-of-state employment among Service 
Delivery Areas. 

This procedure would be in the same spirit as the current effort to adjust for response biases in 
the telephone survey. The procedure would include a statistical approach to the out-of-state 
employment issue. A more direct approach would be interstate data sharing, as envisioned in 
the regional option included in Recommendation 1. 

Finally, the NCEP would like to stress the importance of technical assistance by the DOL in 
facilitating the earliest effective use of the UI wage records for JTPA perfon iance management. 



RECOMMENDATION 5: The Department of Labor should provide 
technical assistance to the States as needed with regard to both the linkage of 
UI and JTPA records and the a- "lysis of program outcomes using the linked 
data. 

This technical & stance could be designed and tested during the conduct of the pilot projects 
envisioned in Recommendation 1. DOL should develop a national mechanism whereby the 
results of these pilot projects would be documented and synthesized foi dissemination to all the 
States. 

In addition, the technical assistance could encompass the development of state-specific 
adjustment models based on individual data. Research efforts by DOL and the States may show 
that such models are a better way for States to prescribe variations in the standards than the 
current national model based on SDA-aggregate data. 



Endnotes 



1 . Section 106 of the Job Training Partnership Act mandates the establishment of performance 
standards. For the adult programs funded under Title II- A, jvrformance is to be measured in 
terms of increases in employment and earnings and reductions m welfare dependency. Section 
1 06(b)(3) provides for access to earnings records, State Employment Security records, and similar 
administrative data. 

2. Such savings to the States are examined in Appendix LB of the project report below. 

3. See Chapter II of the project report below. 

4. Many of these issues are outlined in Nancy Bross, "Using Unemployment Insurance 
Wage-Record Data for JTPA Postprogram Performance Standards," Research and Evaluation 
Associates, Inc., July 1991. This paper was prepared for the US. Department of Labor to facilitate 
discussion at a UI Technical Workgroup meeting in Washing ten, D.G, July 9-10, 1991. 

5. The national methodology for performance standards is optional for the States but widely 
used. States may prescribe variations in the standards based upon state-specific factors. It is 
assumed that States would require at least one year to develop the variations in the standards 
made possible by the UI data. 

6. The pilot projects envisioned in Recommendation 1 are a natural extension of the activities 
involved in the NCEP Project. 

7. SDA-aggregate data cannot generally be used to develop state-specific models due to sample 
size limitations and loss of information content in the aggregation process. 

8. The issue of out-of-state employment was examined in the Canvass of States and the 
Technical Feasibility Study, and it was a prime topic at the NCEP Project Workshop. While 
out-of-state employment raises serious UI coverage problems for a relatively small percentage 
of SDAs, it frequently raises political problems due to local concern over equity in the 
distribution of State incentive grants. 

9. For those States choosing to use the UI data, the basic outcome measure presumably would 
be UI earnings during the first (or second) quarter after the quarter of termination. In contrast, 
for those States continuing to use the survey data, the corresponding outcome measure would 
be reported earnings during the 13th week after the date that participants leave the program, 
assuming the survey approach stays the same. 

10. DOL's proposed regulations on confidentiality of UI data, published for comment on 
March 23, 1992 (57 FR 10064), provide some guidelines for the development of these agreements. 
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USING UNEMPLOYMENT INSURANCE 
WAGE-RECORD DATA FOR JTPA 
PERFORMANCE MANAGEMENT 

PROJECT REPORT 



■ PREFACE 



The National Commission for Employment Policy began this project in 1 988 when eleven States 
joined the Commission in a feasibility study of using the States' Unemployment Insurance (UP 
wage records to track the employment and earnings experiences of participants in programs 
funded under the Job Training Partnership Act (JTPA). The State of Illinois, with Northern 
Illinois University as its agent, served as a clearinghouse for integrating the data from the States. 

The Commission issued a Research Report on the results of its initial investigations, A Feasibility 
Study of the Use of Unemployment Insurance Wage-Record Data as an Evaluation Tool for 
ITPA: Report on Project's Phase I Activities. (January 1991). Because of the encouraging nature 
of the report's findings, in conjunction with growing interest by States in using the UI wage data 
for JTPA performance management, the Commission expanded the project in two phases. As 
a result, twenty States were participating by the end of 1991 (see list of States below), and data 
were being collected for three program years. The U.S. Department of Labor provided financial 
support for the last five States joining the effort. 

The project's expansion also included the three studies presented in this report. The findings 
from these three studies, plus the initial study just cited, formed the basis for the Commission's 
recommendations with regard to using UI wage records for national performance standards, 
the central component of JTPA's system for performance management. These 
recommendations were made by the Commission as part of its ongoing mandate to advise the 
Secretary of Labor on the development of these standards. Commissioners Henri 
Rauschenbach, Melanie Lackland and J. Michael Levesque served as a special workgroup, with 
Mr. Rauschenbach as Giair, to draft these recommendations. 

Findings from the three studies were reviewed and discussed at a Commission Workshop held 
in Alexandria, Virginia on September 5-6, 1991. The workshop served as a forum for the 
representatives of the twenty participating States to offer their views on the feasibility and 
desirability of using the UI wage records for JTPA performance management. Also attending 
were the Commission's workgroup and staff from the US. Department of Labor, the National 
Occupational Information Coordinating Committee, and the Interstate Conference of 
Employment Security Agencies. Following the workshop, the authors of the studies and the 
Commission solicited additional reviews from numerous agencies and experts. 

Ray Worden, formerly of the Commission staff, launched the project and coordinated Phase I. 
Upon the departure of Mr. Worden, Vince Geraci became the project officer. Mr. Geraci is on 
assignment to the Commission from the University of Texas at Austin through an 
Intergovernmental Personnel Act agreement. He completed Phase I and managed all 
subsequent aspects of the project, including the further phases of data collection and the three 
studies. He is also the editor of this report. At various stages he was assisted by Carol Romero 
and other members of the Commission staff, including Robert Ainsworth, Everett Crawford, 
Amy Howard, Barbara Oakley, and Anita Smith. 
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CHAPTER I 

JTPA Performance Standards And Measures 

The Congress recognizes that job homing is an investment in human capital and not 
an expense. In order to determine whether that investment has been productive, the 
Longress finds mat - 

Q) it is essential that criteria for measuring the return on this investment be 
developed; and 

Q) the bask return on the investment is to be measured by the increased employment 
and earmngs of participants and the reductions in welfare dependency. 

Section 106 (a) Job Training Partnership Act 
Public Law 97-300, 1982 

A Brief History Of JTPA Performance Standards 

^l^^^^S ACt P TA) P rovides the ^on's primary programs for training 
economically disadvantaged youth and adults and for training dislocated workers. Titiell-A 

^rTZ^ZTT*** ^ 1982 ' ^T"® WaS V6ry d ™**>ut its intent: the training programs 
™™* 88 an investment in people, and program performance was toteevahuued 
£ ^ ° n tWs investment - ^ "outcomes" orientation, complete with provisions 

for financial incentives and sanctions, would drive this major domestic sodalprqgram 

terf^ s t^Int ° f ** UA De P artmert of Labor (DOL) to e tablish standards 
le^trn^^^ DOL interprets these standards as numerical 

^til^^^n P erfonnano ? with n^pect to program outcomes. For adults in 
tne i itle ll-A programs, the Congress stated that 

... the Secretary shall prescribe standards on the basis of appropriate factors 
whjch may include: (A) placement in unsubsidized employment ^ 
retention in unsubsidized employment, (C) increases in earnings, including 
hourly wages, and (D) reduction in the number of individuals and families 
receiving cash welfare payments and the amounts of these payments. 2 

^^fSZft indkatCd *"* stendards were also to be established for Title 111 programs, 
svs e^hL ^ SU W [&i ' °" e rcaSOn was * at tiie employment and tSiTg 

J^l^n Iess ^Penence with programs for dislocated workers than programs for 
eronomically disadvantaged persons. The JTPA system was given additional time 1 develop 
and test performance standards for Tide III programs. develop 
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Congress suggested a variety of ways to collect the data needed to determine if the performance 
standards were being met: 

The standards shall include provisions governing - 

... cost-effective methods for obtaining such data as is necessary to carry out 
this section, which, notwithstanding any other provision of law, may include 
access to earnings records, State employment security records, Federal 
Insurance Contributions Act records, State aid to families with dependent 
children records, statistical sampling techniques, and similar records or 
3 



measures.' 



Under its mandate to advise the Secretary of Labor on the developmentof JTP A jvrformance 
standards, the National Commission for Employment Policy (NCEP) began a project [1988 to 
assess the feasibility and desirability of using the States' Unemployment I^a^(W^ 
records to track the employment and earnings experiences of JTPA j^dpan ts. J^«port 
presents the findings from three studies designed to address the ad ^^* v ^^^ 
Lues involved in using the UI data for JTPA performance management, ^findings 
underpin the Commission's recommendations for offering States the option to use their UI 

In brief. Chapter II discusses the findings from a canvass of all States J^t 
administrative issues involved in States' using the UI wage records for JTPA P^rmam* 
management. Chapter HI focuses on confidentiality, which is a key issue in using Wdatefo 
non-UI purposes. Chapter IV provides a comparison of the UI wage records to the current 
telephone survey and illustrates their utility for tracking the post-program employment and 
earnings of JTPA participants. 

These chapters necessarily include detailed discussions of specific administrative andtechrdcal 
matters. The remainder of this chapter places these discussions into context by first describing 
JTPA's performance standards, the central component of its system for performance 
management, and then presenting an overview of the issues to be addressed. 

Performance standards in JTPA programs hold the employment and toirung system 
accountable for the programs' outcomes. 4 Each level of the JTPA system - federal, State, and 
local - has a role to play in the performance standards system. 

JTPA directs the Secretary of Labor to establish measures of program performance in terms of 
outcomes and to set standards for these measures. As indicated earlier, the standards are 
numerical levels of minimally acceptable performance. The Secretary may modify these 
standards no more than once every two years. 

For Program Years 1984 and 1985, CETA data were used to set the JTPA standards, since JTPA 
data were not yet available Starting with Program Year 1986, JTPA experience has been used. 
The national measures and standards for program performance have evolved significantly over 
time, as discussed below. 

States have several roles in performance management. First, if the national standards do not 
suffice to meet a State's goals, the Governor may add performance measures and standards 
beyond those established at the national level. Second, the Governor may adjust the standards 
across the State's local Service Delivery Areas (SDAs) in order to take into account variations in 
local conations that affect an SDA's ability to meet its standards but are outside itscontroUfor 
example, the local area's unemployment rate)/ Third, States have a set-aside (6% of their JTI A 
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allocation) which they use to reward SDAs for meeting (or exceed ing) their standards, to provide 
technical assistance to SDAs, and to provide incentives for serving particular "hard to serve" 
groups of participants. 

SDAs and their Private Industry Councils (PICs) are responsible for meeting the performance 
standards. The law gives them great flexibility in the selection and organization of the mix of 
training programs and program participants. An SDA/PIC has strong incentives for meeting 
(or exceeding) its performance standards for Title II-A programs, since at the end of a Program 
Year its State may give it additional funds in the form of "incentive awards," depending upon 
its performance. On the other hand, if an SDA/PIC fails to meet its standards for two 
consecutive Program Years, the State may sanction it. 7 

TITLE II-A (ADULT) PROGRAMS 

DOL began to implement this system of performance standards in Program Year (PY) 1984. 
Since then, information on Title II-A program terminees has been reported by the SDAs at the 
end of a Program Year as part of their TTPA Annual Status Report; these reports have been 
transmitted to the States, and from the States to DOL. 

Initially, the national standards for adult participants were based on the following measures of 
their labor market experiences at the time they left the JTPA program: 

• Entered Employment Rate (the proportion of adult terminees who entered 
unsubsidized employment upon leaving the program); 

• Average Wage Rate at Placement (the average wage rate of those terminees who 
entered unsubsidized employment upon leaving the program); 

• Cost per Entered Employment (the cost of the program per person who entered 
unsubsidized employment); and 

• Welfare Entered Employment Rate (the proportion of adult welfare terminees who 
entered unsubsidized employment upon leaving the program). 

For PY 1986-1987, DOL added "follow-up measures" for the Title II-A (adult) program to its 
existing set Although the JTPA system was required to collect data for these "candidate 
measures," they did not become fully operational as part of national performance standards until 
PY1988. 8 

The follow-up period was the 1 3 weeks after participants left the program, with emphasis placed 
on labor market experiences during the 13th week. Collecting information from the participants 
necessitated the use of a survey, which could be accomplished either by administering a 
questionnaire over the telephone or in person. If the participant could not be contacted by 
telephone, a mail survey could be used as a last resort. 9 Partly due to the expense involved in 
such surveys, DOL allowed States (and SDAs) to collect information on a sample of participants, 
rather than on all participants throughout the Program Year. A statistical procedure for 
determining minimum sample sizes was provided; minimum response rates were required; and 
a method was instituted to adjust, at least partially, for nonresponse biases in the survey. 1 " 

For the Title II-A (adult) program, the follow-up measures were: 

• Follow-up Employment Rate (the proportion of adult respondents who were 
employed during the 13th week after program termination); 
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• Follow-up Weekly Earnings (the average gross weekly earnings of employed 
respondents during the 13th week after program termination); 

• Follow-up Weeks Worked (the average number of weeks worked by respondents 
during the 13-wee'* post-program period); and 

• Welfare Follow-up Employment Rate (the proportion of adult welfare respondents 
who were employed during the 13th week after program termination). 

These measures were, and still are, computed from separate random samples for adult terminees 
and adult welfare terminees. First, an "adult sample" for each SDA is drawn from the universe 
of its adult terminees. Second, a "supplemental welfare sample" is drawn to complete the 
minimum sample size required for adult welfare terminees. 

For PY 1988-1989, national performance standards were set for all of the above eight measures. 
Combined with four measures for the Title 11- A (youth) program, they formed a menu of twelve 
measures from which the Governors were required to select eight for applying the Secretary's 
standards. One of these eight had to be a "quality of placement'* measure for adults (wage at 
placement or weekly earnings at follow-up). This cycle of performance standards was a 
transition period that enabled the JTP A system to shift to full reliance on follow-up measures of 
program effectiveness. 

For the FY 1990-1991 cycle, DOL no longer used the four immediate outcomes measures in its 
performance standards system for Title H-A (adults). The focus shifted fully to participants' 
employment and earnings 13 weeks after they had left the program Governors were required 
to use the following post-program measures: 

• Follow-up Employment Rate; 

• Follow-up Weekly Earnings; 

• Welfare Follow-up Employment Rate; 

• Welfare Follow-up Weekly Earnings. 

This evolution of performance standards reflected the importance that policymakers placed on 
long-term effectiveness of JTPA's training programs. Policymakers were concerned that 
measures of participants' status at the time of their termination from the program had led the 
system to emphasize immediate placements at the expense of long-term gains in participants' 
employment and earnings. The shift to post-program standards was DOL's response: 

Postprogram data provide a more direct measure of long-term employability . 
The .... I postprogram] measures for adults and adult welfare recipients send 
an explicit policy signal that JTPA is a value-added program which generates 
long-term employment for its participants. 11 

TITLE IH/EDWAA PROGRAMS 

Performance measures for the Title III programs for dislocated workers ha veevolved in the same 
general direction as those for the Title H-A (adult) programs, but at a different pace. Several 
impediments have slowed the development of a fully developed performance standards system 
for the Title HI programs: (1) wide variation among States in delivery systems, (2) lack of 
legislation related to rewards and sanctions based on performance, and (3) lack of appropriate 
data. 
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Implementation of Title HI standards also has been affected greatly by the changes EDWAA 
brought to the delivery of services to dislocated workers as of PY 1989. Prior to EDWAA, 
dislocated worker programs were administered at the State level. Post-program information on 
Ti'Je 10 terminees was collected on a sample drawn from the universe of Title HI terminees 
throughout the State. 

As a result of EDWAA, administration of dislocated worker programs shifted more to the local 
level. Governors were required to establish sub-state areas within which Title III /EDWAA 
sub-state grantees and administrative entities were designated. The law gives automatic 
designation to sub-state areas of 200,000 people or more (the same criterion used for SDAs for 
Title D programs). In nearly all States the SDAs have been designated as the sub-state grantees 
and administrative entities for Title Hl/EDWAA programs. 

For purposes of this chapter, the important point is that the accountability for performance 
shifted in large part from the State to the local level. Samples of Title III terminees were now to 
be drawn from eich sub-state area rather than from the State as a whole. This greatly increased 
the data collection effort. 

During the first performance standards cycle, PY 1984-1985, two candidate measures were 
developed for the Title in programs. They were comparable to those of Title II-A (adult) 
programs: Entered Employment Rate and Average Wage Rate at Placement. 

During the next performance standards cycle, PY 1986-1987, DOL required States to collect data 
on these two candidate measures of performance. A national goal was set for the entered 
employment rate; however, no national standards were established at this time. Also beginning 
in PY 1986, DOL required data collection for Title III terminees on three post-program measures: 
Follow-up Employment Rate, Follow-up Weekly Earnings, and Follow-up Weeks Worked. 

The follow-up information was collected for a sample of a State's Title III terminees, as 
mentioned earlier. Data collection procedures for Title III programs emulated those for the Title 
n-A (adult) programs, including sampling strategy, telephone survey method, minimum 
acceptable response rates, and adjustments for nonresponse bias. While adopting the survey 
approach for Title III performance standards, DOL recognized the potential usefulness of UI 
wage-record data: 

In part, the desire for additional data items and for longer-term, postprogram 
tracking might be more easily met for Title III terminees, compared to those 
from Title IIA, by linking to administrative records, especially UI claimant and 
UI wage records. 

However, DOL could not implement a national system of performance standards based on UI 
wage records, since many States did not have the required UI data at that time. 

In PY 1989 (the middle of the PY 1988-89 cycle), two important developments occurred in the 
performance management system for dislocated worker programs. First, a national standard 
was set for the entered employment rate, which has continued into PY 1990-91. (In addition, 
DOL encouraged Governors to set a goal for the average wage rate at placement) Second, due 
to EDWAA, the required collection of both the immediate outcomes and post-program 
outcomes data shifted to the sub-state areas, as indicated earlier. 

Table 1 presents a summary of the evolution of flTA performance measures. An entry of "D" 
denotes that DOL required data collection of the given measure during the designated program 
cycle but did not set a national standard. An entry of M S" denotes that DOL required data 
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collection of the given measure and used it to establish a national standard, i.e., a numerical level 
for the measure which represented minimally acceptable performance. 



TABLE 1 

EVOLUTION OFJTPA PERFORMANCE MEASURES 



"D" = Data Collection Required; "S" = National Standard Established 



PERFORMANCE MEASURE 


PROGRAM YEARS 


SMS 


8647 


nun 


90-91 


Title n*A ( Adult) Jrograot 










entered crnpioyinem Kate 


s 


s 


5 


L? 


Average wage lulls a 1 1 laceinent 


s 


5 


c 


U 


V-USi pe* cnw?reu CMijMvyiiieiu 


s 


5 


b 




Wciiarc cnicrevi cinpiuyiucxii i\4i*v 


s 


S 


s 


D 


Follow-up Employment Rate 




D 


s 


S 


Follow-up Weekly Earnings 




D 


s 


S 


Follow-up Weeks Worked 




D 


s 


D 


Welfare Follow-up Employment Rale 




D 


s 


S 


Welfare Follow-up Weekly Earnings 




D 


D 


S 


Title m/EDWAA Program(a) 










Entered Employment Rate(b) 




D 


S 


S 


Average Wage Rate at Placement 




D 


D 


D 


Follow-up Employment Rate 




D 


D 


D 


Follow-up Weekly Earnings(c) 




D 


D 


D 


Follow-up Weeks Woriced 




D 


D 


D 



(a) Asa result of EDWAA, program performance has been measured at the substate level since PY 1989. 

(b) A standard became effective In PY 1989. From PY 1986-88, DOL had set a national goal, but not a standard for the entered 
employment rate. 

(c) For EDWAA, the average hourly wage rate of employed respondents during the 13th week to being collected. 



Issues In Using The UI Wage-RecanI Data 



By the end of the 1980s there was a confluence of events that led the National Commission for 
Employment Policy (NCEP) to ask if other data sources were available that could reasonably 
substitute for the telephone survey being used to track the employment and earnings of JTPA 
participants after they leave the program. 

First, experience was showing that the telephone surveys for the Title II-A (adult) programs 
were expensive. Second, for the Title III /EDWAA programs the expense of telephone surveys 
had increased significantly due to the shift from State sample surveys to sub-state sample 



surveys. Funds for JTPA as a whole were not increasing, and funds for these surveys were 
coming from administrative dollars, for which there were many competing uses. 

Third, as a result of tire Deficit Reduction Act of 1984, effective September 1988, the employers 
in each State were required to submit quarterly wage reports to a State agency (which may be 
the agency administering the State's unemployment compensation law). Nearly all of the States 
were meeting this mandate through their I H wage reporting system. This data source had not 
been available when DOL developed itscurr Tit approach to performance standards; but by 1988 
(when the Commission began this project), it provided a promising alternative to the survey. 

Fourth, policymakers had become increasingly concerned about the long-term effectiveness of 
JTPA's training services. The survey's 13th-week snapshot of employment and earnings 
provided useful but incomplete information. 

At theoutset of the Commission's project, the UI wage records appeared to offer several potential 
benefits relative to the survey. One was cost-effectiveness: 

• Because the UI data were resident on State computer systems, they appeared to be less 
expensive for State JTPA programs to access than data newly gathered through 
telephone surveys, which are labor intensive. 

Other potential benefits related to program management and evaluation: 

• UI wage records could be used to track the employment and earnings of individuals 
over a substantially longer period than 13 weeks. This could provide more 
comprehensive measures of long-term program effectiveness. 

• The employment and earnings outcomes of all JTPA terminees who obtained jobs 
covered by the UI system, rather than just a sample of terminees, could be examined. 
This could permit SDAs to assess more accurately the performance of individual 
service providers, using the same yardstick that was being applied to them by the 
States under DOL guidelines. 

• With UI data, evaluations of JTPA would (1) not depend upon a person's memory 
about his/her employment and earnings history and (2) not require planning and 
design phases, which can be both time-consuming and costly. These are drawbacks 
to both surveys and large-scale experiments. 14 

In order hat the JTPA system could take advantage of these benefits, the Commission launched 
a multi state examination of the feasibility of Unking participants' records maintained by a 
State's JTPA system with the Stete's wage records stored in its UI system. 15 The project started 
with eleven States in 1988, and expanded to twenty States by 1991. 

INITIAL FINDINGS 

The Commission issued a report on its initial findings in 1991. 16 The report addressed several 
issues: 

• the technical feasibility of linking the UI wage records of JTPA terminees with other 
data collected during their program participation; 

• the administrative feasibility of linking these two data sources, including issues of 
confidentiality and data security; 
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• the content, quality, and applicability of the two data sources for performance 
standards purposes; and 

• the costs of obtaining the two types of data. 

The study's findings were based on integrated data from the eleven original States for PY 1986. 

The study demonstrated that it is feasible for many States to accomplish the required Unking of 
the JTPA and Ul data. Thus, from a technical perspective, the UI data are an alternative to the 
current survey data for performance standards purposes. 

The study also raised several issues mat necessitated further examination. Some were related 
to the administrative feasibility of using UI wage records. 

• Dc State agencies have the capacity to forge and implement the .leeded data-sharing 
agreements? 

• Do States have a way to assure that confidentiality of wage records is maintained when 
data are shared? 

17 

• What are the cost savings associated with the use of Ul wage records? 

• Is it possible for the JTPA system to access the Ul wage records in a manner that j| 
timely in the context of States' need to reward (sanction) SD As for their performance? 

Other issues concerned technical differences between the follow-up surveys and the UI wage 
records in measuring JTPA program performance. One reason for the differences was the 
presence of nonresponse biases in the survey. For example, respondents to the survey had 
higher post-program employment rates than those who were not surveyed. This produces an 
upwardly biased estimate of the employment effects of the training programs. 

Another reason for differences between the two data sources was that some participants obtain 
jobs in States other than where they were trained. Because their earnings would be reported in 
another State, such individuals 'look like" they are not employed according to the UI wage 
records in the State where they were trained. This produces a downwardly biased estimate of 
the employment effects of the training programs, especially forSDAs located near State borders. 

ISSUES COVERED IN THIS REPORT 

Based on the initial findings, the Commission determined that for the UI data to be useful to the 
JTPA system, a number of administrative and technical issues needed to be examined more 
closely. This, along with the growing interest by States in using the UI data for JTPA 
performance management, led the Commission to expand its project. This report contains the 
results of the new studies. 

The key issues fait into four broad categories: (1) State administrative context, (2) data content 
and quality, (3) confidentiality and data access, and (4) application to performance management. 
Table 2 presents a summary, which is cross-referenced to the coming chapters. 

Chapter II examines administrative aspects of linking the two data sources; it is based on 
information from a canvass of all States conducted by the National Governors' Association. 
Discussed are: (1) the extent to which the State agencies responsible for maintaining Ul wage 
records have agreements to share these data with other agencies, JTP A-related or otherwise; (2) 
the staff resource and cost constraints and other barriers that impede the linking of the two data 
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bases; (3) methods that States have developed to overcome these barriers and constraints; and 
(4) the potential cost savings associated with use of UI wage records. The chapter concludes 
with a comparison of the content and applicability of UI wage records and the current telephone 
surveys for purposes of program management and evaluation. 

Chapter III focuses on the need to maintain the confidentiality of wage-record data when these 
data are used for non-UI purposes. It assesses the confidentiality provisions of States' 
unemployment compensation statutes and related administrative documents. 

Chapter IV is a continuation of the Commission's technical feasibility study. It presents the 
findings from Phase II which covered fifteen States and Program Years 1986 and 1987. The 
chapter analyzes nonresponse biases in the follow-up survey and coverage problems in the UI 
data. It also uses the UI data to track the labor market experiences of JTPA terminees in the first 
two years after they left the program. 

Findings from these studies were reviewed and discussed at a Commission Workshop, "Using 
Unemployment Insurance Wage-Record Data for JTPA Performance Management;' held in 
Alexandria, Virginia on September 5-6, 1991. (Appendix LA contains the workshop agenda and 
list of participants.) The comments and suggestions from the participants at the workshop have 
been incorporated into the current chapters, 

TABLE 2 

KEY ISSUES IN USING UI WAGE-RECORD DATA FOR 
JTPA PERFORMANCE MANAGEMENT 

State Administrative Context (Chapter II) — 

1. Current uses of State UI data by JTPA and other programs 

2. Location of JTPA and Li agencies within State structure 

3. Centralized vs. decentralized systems of JTPA follow-up 
Data Content and Quality (Chapters ILIV) 

1. UI coverage of Jobs and participants 

2. Slippage due to out-of-state employment 

3. Items not included in UI wage records 

4. Data validity, accuracy, consistency 

5. Match rates of UI and JTPA records 
Confidentiality and Data Access (Chapters U HLIV) 

1. Confidentiality and data security 

2. Slate capacity to perform record linkages 

3. Developmental and continuing costs 

4. Data Sharing Agreements 

Application to Performance Management (Chapters ELI V) 

1. Timeliness with regard to rewards and sanctions 

2. Equity of performance standards between States 

3. Equity of performance standards within State 

4. Relation to program management at SDA level 

5. Technical assistance needs 
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COST SAVINGS 



At the workshop, further questions arose with regard to the cost savings associated with use of 
UI wage records, discussed in the Commission's first report on the project (cited earlier) and 
Chapter II of this report. The State representatives reported wide variations in these potential 
savings. As a result, with the assistance of the States, the Commission undertook a further 
analysis of potential savings in administrative costs associated with the use of UI wage records. 

Table 3 gives an overview of the results based on nineteen States for PY 1990, It shows (1) the 
actual average annual costs of the follow-up survey for JTPA Title Il-A (adult) and Title HI 
programs combined; (2) the estimated average annual costs of using UI wage records, and (3) 
the estimated average annual savings associated with replacing the survey data with UI 
wage-record data. It also shows the States' minimum and maximum values for these categories 
of costs and savings. 

The data in Table 3 confirm the potential size of the annual savings associated with use of UI 
wage records rather than telephone surveys: an average of $102,000 for the nineteen States 
involved in this analysis. The data also confirm the wide range across States in potential savings: 
from $4,500 to $300,000 annually. Appendix LB discusses possible reasons for this variation: 
differences among States in the size of their JTPA program, in their survey response rates, in the 
entity conducting the survey, and in the scope of the survey. 

In conclusion, the UI wage-record data offer substantial cost savings to the JTPA system as well 
as other benefits in managing program performance. The next three chapters provide detailed 
discussions of the administrative \nd technical issues involved in taking advantage of this data 

resource. 



TABLE 3 

ESTIMATED ANNUAL COSTS AND SAVINGS PER STATE: 
Telephone Surveys Versus UI Wage Records 



Cost/Savings Category 


Average 


Minimum 


Maximum 


Cost of Titles II-A&ni 
Follow-up Survey 


$125,320 


$8347 


$371,450 


Estimated Cost of Using 
UI Wage Records (a) 


$23,290 


$1,292 


$77,692 


Potential Savings Associated with 
Replacing Survey with UI Wage Records (b) 


$102,030 


$4^71 


$301,872 



(a)Estimoted cost equals total number of a State's PY 1990 tenninees multiplied by S2.00. See Chapter U for development of 
this $2.00 benchmark. 



(b)Esbmated saving equals the difference between each State's actual cost of using the survey and estimated cost of using the 
UI wage-record approach. 
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Endnotes to Chapter I 



1. In August 1988 Title m was amended by the Economic Dislocation and Worker Adjustment 
Assistance (EDWAA* Act. The Act, which became fully operational on July 1, 1989, significantly 
altered the delivery system for dislocated worker programs. As a result, Title III programs have 
become known as "EDWAA programs." 

2. JTPA, Section 106 (b)(1). Because UI wage records are not being proposed for use in 
performance standards for Title II- A (youth) programs, these standards are not discussed in this 
report 

3. JTPA, Section 106 (b)(3). As discussed in the subsequent chapters of this report, the States' 
UI systems are a promising source of "earnings records" for JTPA purposes, 

4. For a more detailed discussion, see SRI, ITPA Performance Standards: Effects on Clients, 
Services and Costs, National Commission for Employment Policy, Washington, D.C, Research 
Report Number 88-16, September 1988. 

5. Although not mandated to do so, the US. Department of Labor (DOL) also has developed 
a multiple regression methodology which States may use to adjust their SDAs' standards for 
variations in local client mixes and economic conditions. 

6. A Program Year begins July land ends June 30. 

7. States have latitude in the way they structure their system of incentive a wards and sanctions. 
For a discussion of various ways States have handled, and can handle, their incentive awards 
system, see SRI, Developing Effective JTPA Performance Standards Incentive Policies: A 
Technical Assistance Guide, National Commission for Employment Policy, Washington, D.C, 
Research Report Number 89-07, March 1990. 

8. U.S. Department of Labor, Employment and Training Administration, Follow-up Technical 
Assistance Guide for PostProgram Data Collection Under the Job Training Partnership Act, 
Version 3.0, Washington, D.C, June 1986. 

9. In some States the surveys are conducted at the State level, either by staff or by outside 
contractors; in other States, the SDAs are responsible for conducting the surveys. 

10. Minimum response rates of 70% were required for four of the following six groups: adults 
employed at termination; adults not employed at termination; adult welfare recipients 
employed at termination; adult welfare recipients not employed at termination; dislocated 
workers employed at termination; and dislocated workers not employed at termination. These 
requirements applied to each SDA for the Title II-A program, and to each State for the Title in 
program See VS. Department of Labor, Follow-up Technical Assistance Guide, June 1986, p. 



11. Federal Register, Vol. 55, No. 4, January 5, 1990, p. 515. 

12. US. Department of Labor, Follow-up Technical Assistance Guide, June 1986, p. 61. 

13. The vast majority of jobs are covered by theUI wage-record system. Exceptions include 
self-employed workers, workers on small farms, some domestic workers, federal civilian and 
military employees, railroad employees, and employees of religious organizations. See the 
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discussion in John Baj and Charles E Trott, A Feasibility Study of Jie U se of Unemptovtnent 
Insurance Waee-Record Data as an Evaluation Tool for 1TPA: Rep ort on Projecfs Phase I 
Activities, National Commission for Employment Policy, Washington, D.C, Research Report 
Number 90-02, January 1991, pp. 8-10. 

14. DOL is conducting a arge-scale experiment designed to evaluate the net impact of JTPA 
through a comparison of the employment and earnings outcomes of selected groups of JTPA 
participants with the employment and earnings of randomly selected control groups; see Fred 
Doolittle and Unda Traeger, "Implementing the National JTPA Study/ Manpower 
Demonstration Research Corporation, April 199a NCEP is sponsoring a study by William 
Bowman which is testing the feasibility of using State UI and Job Service records in a 
non-experimental approach to estimating the net impact of JTPA training activities; findings are 
anticipated in Summer 1992. A survey of State evaluation efforts is presented in "JTPA State 
Evaluation Reports," In Brief. National Governors' Association, Washington, D.C, Jury 23, 1990; 
for example, see James S. Hanna and Zina Turney, The Net Impact of the Nevada JTPA Title H 
Program," Nevada Job Training Office, February 1988. 

15. Around the same time, DOL sponsored a meeting to discuss a national approach to 
archiving UI wage records for purposes of general labor market analysis and program 
evaluation. The proceedings were published in The Feasibility of a National Wage Record 
Database, Northeast-Midwest Institute, Washington, D.C, 1989; two contributors to this volume 
noted the potential application of these UI data to performance standards (see Christopher King, 
p. 7, and John Bishop, pp. 18-21). NCEFs approach centers the database development at the 
State, rather than the national, level. 

16. John Baj and Charles E. Trott, A FeasibiUtv Study of the Use of Unemployment Insurance 
Wggs-Record Data as an Evaluation Tool for JTPA: Report on Project's P hase 1 Activities, 
National Commission for Employment Policy, Washington, D.C, Research Report Number 
90-02, January 1991. 

17. No State lin the initial study! estimated that it would cost more than $10,000 per year to 
access UI data for new terminees and to update existing files with additional quarters of 
post-program data. The estimated costs of collecting the post-program survey data are much 
higher, as discussed in the last section of this chapter and also Chapter II. 

18. Under the current performance standards system, outcomes data for four consecutive 
quarters need to be available to State JTPA agencies hist after the end of a Program Year, for the 
States to reward SDA performance on a timely basis. In order to meet this deadline while still 
including four quarters' worth of outcomes under the 13th-week survey approach, the system 
uses a "Follow-up Year" to determine SDA performance: the first three quarters of the most 
recent Program Year and the fourth quarter of the previous one. Use of the UI data might require 
a different definition of the "Follow-up Year." 
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AGENDA 



USING UNEMPLOYMENT INSURANCE WAGE-RECORD DATA 
FOR JTPA PERFORMANCE MANAGEMENT 

Embassy Suites Hotel, Alexandria, Virginia 

September 5, 1991 



Presentation and Group Discussion of Research Findings 

PANELISTS 



TIME 






9:00- 


930 


Opening Remarks 


930- 


1030 


Canvass of the States 


10:30- 


10:45 


Coffee Break 


10:45- 


12:00 


Canvass of the States 


12:00- 


1:00 


Buffet Luncheon 


1:00- 


230 


Confidentiality Study 


2:30- 


330 


Feasibility Study 



3:30 - 3:45 
3:45 - 5:00 



Coffee Break 
FeasibUity Study 
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John Gartland, Chair, and Barbara 
McQuown, Director, National 
Commission for Employment Policy 

Lorraine Amico and Jose Figueroa, 
National Governors' Association 

Jim Hanna, State of Nevada, and Tim 
Harmon, State of Illinois 



David Stevens, University of Baltimore 

Mike Dall, State of South Carolina, and 
Ron Stewart, State of Oregon 

Group Discussion 

John Baj, Sean Fahey and Pete Trott, 
Northern Illinois University 

Douglass Jex, State of Utah, Patrik 
Madaris, State of Indiana, and Sandy 
Owens, State of Missouri 



Group Discussion 



1 i 



September 6, 1991 



TIME 
9:00 - 9:30 

9:30 - 10:30 
10:30 - 10:45 
10:45 - 11:45 
U:45 - 12:00 



Executive Session of State Representatives Workgroup 
TOPIC PANELISTS 
Summary of Issues 



Vince Geraci, National Commission for 
Employment Policy 



Assessment of Issues Group Discussion 



Coffee Break 
Recommendations 
Closing Remarks 



Group Discussion 

John G art land, Chair, National 
Commission for Employment Policy 
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USING UM EMPLOYMENT INSURANCE WAGE-RECORD DATA FOR 

JTPA PERFORMANCE MANAGEMENT 



Embassy Suites Hotel 
Alexandria, Virginia 
September 5 & 6, 1991 

LIST OF ATTENDEES 



Mr. Steve Aaronson 
US. Department of Labor 
ETA/OSPPD, Room N-5310 
200 Constitution Avenue, NW 
Washington, D.C 20210 

Mr, Robert Ainsworth 
National Commission lor 
Employment Policy 
1522 K St., N.W., Suite 300 
Washington, D.C 20005 

Ms. Lorraine Amico 
Center for Policy Research 
National Governors' Association 
444 North Capitol Street, NW, Suite 250 
Washington, D.C 20001-1572 

Mr.JohnBaj 

Center for Governmental Studies 
Northern Illinois University 
DeKalb, Illinois 60115-2854 

Mr. William Bowman 

Annapolis Economic Research 
P.O. Box 562 

Annapolis, Maryland 21404 

Ms. Nancy Bross 
Public Policy Support 
1377 McLendon Avenue, NE 
Atlanta, Georgia 30307 



Mr. Tom Clendenning 

Labor, Employment and Training 
Atkins Building, Suite 201 
1320 Executive Center Drive 
Tallahassee, Florida 32399-0667 

Mr. Carlos Cracraft 
Kentucky Department for 
Employment Services 
275 East Main Street, 2-W 
Frankfort, Kentucky 40621 

Mr. Gary Crossley 
Interstate Conference of 
Employment Security Agencies 
444 North Capitol Street, NW, Suite 126 
Washington, D.C. 20001-1572 

Mr. Mike Dal! 

LMI/JTPA 

Employment Service Commission 
1550 Gadsden Street 
Columbia, South Carolina 29202 

Mr. Sean Fahey 

Center for Governmental Studies 
Northern Illinois University 
DeKalb, Illinois 601 15-2854 

Mr. Jose Figueroa 
Center for Policy Research 
National Governors' Association 
444 North Capitol Street, NW, Suite 250 
Washington, D.C. 20001-1572 
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Mr. Richard Froeschle 
State Occupational Information 
Coordinating Committee 
TEC Building, Room 526T 
15th & Congress 
Austin, Texas 78778 

Mr. John C Gartland 
Chairman 

National Commission for 
Employment Policy 
1522 K St., N.W., Suite 300 
Washington, D.C 20005 

Mr. James Gaston 
JTPA Program 

Texas Department of Commerce 
First City Center 
816 Congress Avenue, Suite 1300 
Austin, Texas 78711 

Mr. Vince Geraca 
National Commission for 
Employment Policy 
1522 K St., N.W., Suite 300 
Washington, D.C 20005 

Ms. Karen Greene 
US. Department of Labor 
ETA/OSPPD, Room N-5621 
200 Constitution Avenue, NW 
Washington, D.C. 20210 

Mr. Jim Hanna 

Employment Security Research 
Department of Employment Security 
500 East Third Street 
Carson City, Nevada 89713 

Mr. Timothy Harmon 

Job Training Programs Division 
Department of Commerce & 
Community Affairs 
620 East Adams Street 
Springfield, Illinois 62701 

Mr. Thomas Hawk 

Bureau of Job Training 

1105 Labor and Industry Bldg. 

7th and Foster Streets 

Harrisburg, Pennsylvania 17120 



Ms. Amy Howard 
National Commission for 
Employment Policy 
1522 K St., N.W., Suite 300 
Washington, D.C. 20005 

Mr. John Huegelmeyer 
Department of Economic & 
Employment Development 
1100 North Eutaw Street, Room 314 
Baltimore, Maryland 21201 

Mr. Douglass Jex 

Office of Job Training 
Department of Community & 
Economic Development 
324 South State Street, Suite 210 
Salt Lake City, Utah 84111 

Mr. Jeff Klein 

Employment Services & Training 
Programs 

Department of Employment 

317 Main Street 

Boise, Idaho 83735-0001 

Mrs. Melanie M. Lackland 
Commissioner 
National Commission for 
Employment Policy 
152 K St., N.W., Suite 300 
Washington, D.C. 20005 

Mr. J. Michael Levesque 
Commissioner 
National Commission for 
Employment Policy 
1522 K St., N.W., Suite 300 
Washington, D.C. 20005 

Mr. Patrik Madaras 

Evaluation Division 

Department of Workforce Development 
10 North Senate Avenue, Room 101 
Indianapolis, Indiana 46204 

Mrs. Barbara McQuown 

Director 

National Commission for 
Employment Policy 
1522 K St., N.W., Suite 300 
Washington, D C. 20005 
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Mail Stop KC-11 
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Ms. Gail Nottingham 

Research Policy & Evaluation Division 
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Department 

4615 West Broad Street, 3rd Floor 
Richmond, Virginia 23230 

Mrs. Barbara Oakley 
National Commission for 
Employment Policy 
1522 K St., N.W., Suite 300 
Washington, D.C 20005 

Ms. Sandra Owens 

Division of Job Development & Training 
Department of Economic Development 
221 Metro Drive 
Jefferson City, Missouri 65109 

Mr. Henri S. Rauschenbach 

Commissioner 
National Commission for 
Employment Policy 
1522 K St., N.W., Suite 300 
Washington, D.C 20005 

Mr. David Rigsby 

Commissioner's Office 
Kentucky Department for 
Employment Services 
275 East Main Street, 2-W 
Frankfort, Kentucky 40621 

Ms. Carol J. Romero 

National Commission for 
Employment Policy 
1522 K St, N.W., Suite 300 
Washington, D.C. 20005 

Ms. Margaret Sharkey 
US. Department of Labor 
ETA/OSPPD, Room N-5310 
200 Constitution Avenue, NW 
Washington, D.C. 20210 



Mr. David Smith 

Texas Department of Commerce 

P.O. Box 12728 

Austin, Texas 78711 

Mr. Robert Snow 

Job Training Division 
Georgia Department of Labor 
148 International Blvd., NE, Suite 650 
Atlanta, Georgia 30303 

Mr. David Stevens 
University of Baltimore 
R.G. Merrick School of Business 
1420 North Charles Street 
Baltimore, Maryland 21201-5779 

Mr. Ron Stewart 

JTPA Information Resource Unit 

Department of Economic Development 
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Mr. Charles Trott 
Center for Governmental Studies 
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US. Department of Labor 
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Mr. Rich West 

SRI, International 
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Mr. James Woods 

National Occupational Information 

Coordinating Committee 

2100 M Street, NW, Suite 156 

Washington, D.C. 20037 

Mr. Ray Worden 

Executive Director 
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Costs Of Follow-up Surveys Versus UI Wage 
Records For Tracking Post-program Outcomes 
Under JTPA 

Introduction 



A major reason for switching to UI wage records is the savings in administrative costs that would 
accrue to the JTPA system. The cost of the follow-up survey for Title II-A (adults) and Title 
IH/EDWAA is approximately $19 per completed interview; in contrast, the estimated cost of 
using UI data iv *e two programs is approximately $2 per program terminee. 1 For the JTPA 
system as a whole, the amount expended for the survey is about $3.1 million; with UI wage 
records, the amount expended would be about $12 million. 2 Thus, according to these estimates, 
the JTPA system could reap annual savings in the neighborhood of $2 million dollars by 
switching to UI wage records in place of the current surveys. These substantial savings in 
administrative costs could be used for other programmatic purposes. 

This appendix presents additional findings on the potential cost savings of using UI wage 
records instead of the current surveys for Title II-A (adult) and Title III/EDWAA programs. The 
wide variation of cost savings among States is documented, and possible reasons for this 
variation are explored. 

The data used in the analysis cover Program Year (PY) 1990. They were requested from the 
twenty States participating in the Commission's project. Nineteen of the States were able to 
provide the necessary information. Given the wide variation in cost savings across these States, 
the following analysis should be regarded only as suggestive of the national experience. 

Table B.l presents the actual average annual costs of the follow-up survey for JTPA Title Il-A 
(adults) and Title III/EDWAA combined, the estimated average annual costs of using UI wage 
records for these programs, and the estimated average annual savings associated with replacing 
the survey data with UI wage-record data. It also shows the States' minimum and maximum 
values for these categories of costs and savings. 

The average annual cost of using follow-up surveys for Title II-A (adults) and Title HI /EDWA A 
participants is $125,320. By comparison, the average annual cost of using UI data for follow-up 
purposes is $23,290. This yields an estimated average annual saving of $102,000 per State 
participating in the Commission's project. Projections of this estimate to States not in the 
Commission's project should be resisted, given the wide variation among States. 4 
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TABLE B.1 

ANNUAL COSTS AND SAVINGS PER STATE 



Cost/Savings Category 


Average 


Minimum 


Maximum 


Cost of Titles H-A & HI Follow-up Survey 


$125,320 


$8347 


$371450 


Cost per Completed Interview 


$2637 


$5.99(a) 


$69.90(b) 


Estimated Cost of Using UI Wage Reeords(c) 


$23,290 


$1,292 


$77,692 


Potential Savings Associated with Replacing 
Survey with UI Wage Records (d) 


$102,030 


$4,571 


$301^72 



(a) This minimum value is an underestimate of the survey cost* This State, which conducted the survey in-house, was not able 
to provide the overhead costs (including telephone charges) involved. 

(b) This maximum value is an overestimate of the survey cost This State, which conducted the survey through an outside 
contract was not abte to separate analytical costs from survey costs. This bundling of costs, which tends to overstate the 
potential cost savings of replacing the survey, occurs for eight of the nineteen responding States. 

(c) Estimated cost equals total number of a State's PY 1990 terminees multiplied by $2.00, 

(d) Estimated saving equals the difference between each State's actual cost of using the survey and estimated cost of using the 
UI wage-record approach. 



Determinants Of Survey Costs 



This section discusses four reasons why States may differ in the cost per completed survey 
interview. They are: (1) the size of their JTPA program, (2) the achieved response rate to the 
survey (3) the entity selected to conduct the survey, and (4) the scope of the survey (for example, 
the inclusion of analytical services). 

SIZE OF STATE 

It was hypothesized that the cost per completed interview would be higher for small States than 
for large States because surveys have fixed costs, regardless of the number of participants to be 
interviewed. The fewer the number of persons interviewed, the fewer the number of persons 
over whom these fixed costs could be spread. 

To examine whether costs per completed interview varied systematically by State size, the 
sample States were ranked according to the total number of Title Il-A and Title IH/EDWAA 
terminees in PY 1 990. States were classified as: 

• "Large" if they had more than 10,000 terminees; 

• "Medium" if they had 5,000 - 10,000 terminees; and 

• 'Small" if they had fewer than 5,000 terminees- 

Table B2 shows both average annual cost and average annual cost per completed interview for 
the three size categories of States- While large States have higher average oj\rual costs, small 
States have higher average costs per completed interview. Small States averaged $35.97 per 
completed interview while large States averaged $19.58. (Medium size States were in the middle 
at $27.40 per completed interview.) From this perspective, the cost advantages of the UI 
wage-record approach are considerable for small as well as large States. 
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While the differences in costs among the three categories of States are sizeable, they are not 
statistically significant. This lack of significance is due to the small number of States and wide 
range of costs per completed interview across States within each of the different size categories. 
This important caveat applies as well to the other cost differences reported below. 

RESPONSE RATES 

A second hypothesis was that the cost per completed interview would eventually rise will, 
increases in the response rate sought by the State. High response rates are more costly to achieve 
because numerous call-backs may be required to make successful contact with harder-breach 
terminees (for example, those who can only be reached through friends or relatives). 



TABLE B.2 

AVERAGE ANNUAL TOTAL COST AND COST PER COMPLETED 

INTERVIEW BY SIZE OF STATE 



Size of State 
Program(a) 


Average Annual 
Total Cost 


Average Cost Per 
Completed Interview 


Large (N=8) 


$196,806 


$19.58 


Medium (N=6) 


$102,455 


$27.41 


Small (N=5) 


$38380 


$35.97 



(aTNg " indicates the number of States in the category. 



To test this hypothesis, States were ranked according to their PY 1990 response rate. Response 
rales were classified as: 

• "High" if more than 80% of the interviews were completed; 

• "Moderate" if between 75% and 80% of the interviews were completed; and 

• "Low" if less than 75% of the interviews were completed. 

The U.S. Department of Labor (DOL) requires a minimum response rate of 70%. However, 
despite considerable effort by the JTPA system, many SDAs do not reach this goal. For example, 
in PY 1987, over 40% of all SDAs had response rates below the 70% minimum, 5 

Table D3 shows the average cost per completed interview for the nineteen States grouped 
according to their level of response rates, In contrast to expectations, States with the highest 
response rates also had the lowest cost per completed interview. Conversely, States with the 
lowest response rates had the highest cost per completed interview. 

In retrospect, this contrary finding may be reconciled in two ways. First, while efforts to achieve 
high response rates may be costly within a given State, differences in response rates across States 
may not explain differences in costs across States given that other factors are at work- Second, 
States may accept low response rates because their average costs per completed interview are 
high; in other words, high survey costs may cause low response rates. For such States, the Ul 
wage-record approach is all the more attractive. 
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TABLE B.3 

COST PER COMPLETED INTERVIEW BY STATES' RESPONSE RATES (a) 



Category of Response Rate (b) 


Average Cost per Completed Interview 


High Response Rate (N=4) 


$18.44 


Moderate Response Rate (N=8) 


$24.36 


Low Response Rate (N=7) 


$33.19 



(a) The Average response rate for the 19 States combined was 76%. 

(b) "N= "indicates the number of States in the category. 



ENTITY CONDUCTING FOLLOW-UP SURVEY 

A third hypothesis was that the cost of he follow-up survey depends upon the entity conducting 
it. Specifically, due to economies of scale, *>wer costs would be associated with a survey that is 
centralized at the State level ratter than decentralized at the SDA level. Lower costs could also 
be associated with a survey that is conducted "in-house" rather than by an outside contractor. 

Table B.4 shows cost per completed interview by the entity conducting the follow-up survey. 
These data suggest that surveys conducted at the SDA level are more expensive than those 
conducted at the State level. This indicates that States with a decentralized follow-up system 
could obtain relatively large cost savings by using the UI wage-record approach. 

The survey is potentially less expensive when conducted in-house. However, the number of 
States in this category (N=2) is far too small to permit generalizations. In addition, these two 
States collected the minimum amount of information, and they included no analytical costs in 
their estimates. ISee also Note (b) of Table B.4.} This highly tentative finding is reported for 
completeness, but no conclusions should be drawn from it. 



TABLE B.4 
COST PER COMPLETED INTERVIEW BY 
ENTITY CONDUCTING THE SURVEY 



Entity Conducting Survey (a) 


Average Cost per Completed Interview 


State-Level 




In-House (N=2) 


$6.92(b) 


Outside Contractor (N=13) 


$26.23 


SDA-Level (N=4) 


$3652 



(a) "NT indicates the number of States in the category. 

(b) One of these two States was not able to provide the overhead costs associated with the survey, as indicated in Note (a) of 
Table B,l More generally, in-house cost estimates may be biased downward due to inadvertent omission of certain overhead 
costs. The other State has an exemplary survey operation, which includes the use of a Watts line to keep costs low. 



SCOPE OF SURVEY 

The final hypothesis was that the scope of the survey would influence the average cost per 
completed interview. A survey could be more or less expensive depending upon ( 1 ) the number 
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of questions asked, (2) the groups of terminees covered, and /or (3) the inclusion of costs for 
analytical services in an outside contract. 

Table B5 shows the average annual survey cost for the nineteen States by whether or not the 
cost includes (1) data analysis as well as data collection, (2) additional questions in the survey 
beyond those required by DOL and (3) more groups of terminees than those required by DOL. 

This information indicates that analytical costs may be a substantial component of a State's 
survey costs. The average cost per completed interview was $32.80 for the eight States that 
included analytical costs in their estimates; it was $21 .69 for the eleven States thatdid notinclude 
analytical costs. These findings support the NCEPs recommendation that in order to reap the 
full benefits of savings from the use of UI wage records, small and /or neighboring States may 
wish to develop a regional consortium to handle analysis of the outcomes data for performance 
management and evaluation. 

By comparison, inclusion in the survey of either additional questions or more terminee groups 
appears to have little effect on the cost per completed interview. The difference in average cost 
per completed interview was about $2.00 between States that did, and did not, have eiiher 
additional questions or additional terminee groups in the survey. 



TABLE B.5 

AVERAGE ANNUAL COST PER COMPLETED INTERVIEW 

BY SCOPE OF SURVEY 



Scope of Survey (a) 


Average Annual Cost Per Completed Interview 


Cost of Survey: 


Includes Analysis (N=8) 


$32.80 


Does Not Include Analysis (N=ll) 


$21.69 


Survey Includes: 


More Questions than Required (N=16) 


$25.99 


No Additional Questions (N=3) 


$28.39 


Survey Covers: 


More Groups than Required (N=8) 


$27.10 


No Additional Groups (N=ll) 


$23.83 



(a) *N= " indicates the number of States in the category. 



WHO BENEFITS FROM THE SAVINGS? 

States finance the follow-up survey in many ways. The different methods were combined into 
three categories: (1 ) States in which the survey was financed entirely by State-level JTPA funds, 
(2) States in which the survey was financed entirely by SDA-level funds, and (3) States in which 
the financing was accomplished through a combination of State-level and SDA-level funds. The 
numbers of States falling into these categories were eight, seven, and four, respectively. In 
conclusion, the dist.ibution of the potential savings in administrative costs from using the UI 
wage records in placeof the current telephone survey is likely to vary considerably across States. 




Endnotes To Appendix LB 



1 . The sources of these estimates of unit cost are: John Baj and Charles E. Trott, A Feasibility 
Study of the Use of Unemployment Insurance Wage-Record Date as an Evaluation Tool for 
TTPA: Report on Protects Phase 1 Activities, National Commission for Employment Policy, 
Washington, D.C., Research Report Number 90-02, January 1991; and Chapter II of this report 
These cost estimates were discussed, and generally agreed upon, at a Commission-sponsored 
workshop on September 9-10, 1991. The workshop participants included representatives from 
seventeen of the twenty States that have been participating in this Commission project. 

2. The source of these cost estimates is Chapter II of this report. 

3. The responding States were California, Florida, Georgia, Illinois, Idaho, Indiana, Kentucky, 
Maryland, Missouri, New Hampshire, Nevada, Oregon, Pennsylvania, South Carolina, South 
Dakota, Texas, Utah, Virginia, and Washington. Arizona was not able to provide the cost data 
because of the decentralized nature of its follow-up system. 

4. Also, this average is influenced heavily by a few high values reflecting unusual state 
experiences. The median saving, a more robust statistic for st h a small sample, is $81,349. 

5. Richard W. West, Katherine P. Dickinson and Catherine M. Casserly, Performance 
Standards '89: Managing Quality Programs, Follow-up Training Materials, Menlo Park, 
California: SRI International, 1989. However, VOL has informed NCEP that the response rates 
have improved markedly since PY 1987. Recent estimates indicate that only 20% of all SDAs 
had response rates below 70% in PY 1990. 
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■ CHAPTER II 

State Capacity To Share Wage Records: 
The JTPA Experience 

Background 



Increasing concern with the accountability of federally funded programs has resulted in 
increasing emphasis on performance-based systems. Title II-A of the Job Training Partnership 
Act (JTPA) has a highly developed performance-based system in which national performance 
standards play a key role. These standards have evolved steadily over time. When JTPA was 
implemented in 1983, adult and youth programs were evaluated at the time the participants left 
the program. Measures of employment and wages were used for both adults and youth; in 
addition, other positive nonemployment outcomes were used for youth. 

In Program Year (PY) 1988, the VS. Department of Labor (US. DOL) included four adult 
post-program measures in its set of performance standards. States and local areas were required 
to obtain post-program data through direct contact with the program participants. The data 
were collected by telephone survey in the 13th week after participants terminated from the 
program. By PY 1990, the adult outcome measures obtained at the point of termination were 
eliminated in favor of the post-program measures obtained from the survey. 

The use of databases external to JTPA, such the States' Unemployment Insurance (UI) wage 
records, has been proposed as an alternative method for collecting post-program data on former 
participants. Numerous State agencies have shared the wage records under various conditions 
for such purposes as research, accountability and law enforcement. 

The use of these records for JTPA performance standards has the potential for solving a number 
of data collection problems the States have encountered. These include the difficulty of meeting 
federally required response rates for the telephone survey, cost constraints of the post-program 
data collection, and concerns about data accuracy. However, as discussed in Chapter I, a 
number of administrative and technical issues have to be resolved in order to take full advantage 
of these wage data. 

In 1988, when post-program measures were included in national performance standards under 
JTPA, the National Commission for Employment Policy (NCEP) initiated a project to examine 
the issues involved in using the States' wage-record data to assess the effectiveness of JTPA 
programs. The NCEP invited all States to apply for participation in the project, and 
twenty-seven States expressed interest. Eleven States were selected on the basis of their having 
adequate data on JTPA services provided to JTPA terminees during PY 1986, and their ability 
to access UI wage records on the employment experiences of these terminees both before and 
after they participated in the program. 1 The project was later expanded; Phase II added four 
States and Phase III, five more States. In all, twenty States have participated in NCEP's project. 

Results of a survey conducted in 1989 by the Center for Policy Research of the National 
Governors' Association (NGA) revealed that NCEP's project had been instrumental in assisting 
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the original eleven participating States to make greater use of these data. However, the survey 
responses also showed that wage-record data were being used to a relatively limited extent 
despite improvements in them during the 1980s, which had widened their potential applicability 
for statistical purposes. The limited applications of wage-record data for JTPA program 
management and evaluation as of 1989 were most likely related to several long-standing issues 
regarding interagency and interstate data sharing. These issues are taken up in the present 
study. 

Two key factors have raised States' interest in using their wage-record data for accountability 
purposes in JTPA and other employment and training programs. The first is the availability of 
quarterly wage data in all States. Prior to 1984, about three-fourths of the States were collecting 
these data for purposes of administering their UI benefit programs. However, the Deficit 
Reduction Act of 1984, effective September 1988, required all States to collect information on 
individual earnings for purposes of detecting fraud and abuse in federally funded programs. 
In response, all States now collect quarterly wage data, either in the State Employment Security 
Agency (SESA) or in another department, such as Revenue or Taxation. 

While the wage record systems of the States have emerged over the past few years as a powerful 
tool for purposes of program management, research and evaluation; the sole purpose for 
Congressional funding for SESAs to collect the wage-record data is administration of the UI 
program. States that have devoted considerable resources to expand and improve the quality 
of the UI data, particularly about business establishments, have stimulated the interest of non-UI 
agencies in the use of these date for statistical purposes. These external applications, however, 
are often viewed by SESAs as non-essential, time consuming and disruptive to UI administrative 
processes. 

The second key factor is the rapid rate of technical progress in information processing. Evolving 
computer technology has encouraged a shift to the maintainence of micro data rather than 
aggregate data. In addition, new transmission media will allow future data to be available more 
readily than today's capabilities allow. 3 Such technical progress has created an environment 
for improved access to secondary data sources, such as the administrative records of other 
agencies. / 

/ 

The potential for ready technical access to administrative records has increased the need to 
achieve a balance between protecting confidential data and allowing reasonable access through 
data sharing agreements. The confidentiality issue has evoked more concern among date 
producers and users than any other single issue. Confidentiality provisions restrict access to UI 
data in certain areas but do not preclude all uses of the data. The scope of access to these data, 
and the specification of needed safeguards, have been subject to considerable debate over the 
years. 

I.. 1990, NCEP sponsored the NGA Center for Policy Research to examine the confidentiality 
and other issues involved in the potential use of UI wage records for accountability purposes 
within the JTPA program In addition, the Office of Adult and Vocational Education in the US. 
Department of Education and the National Occupational Information Coordinating Committee 
are co-sponsoring the NGA to conduct a companion study to examine data sharing 
arrangements between wage record offices and vocational education offices. A report on this 
further work will be issued in early 1992, 

The JTPA part of the study had three goals: 

* to document current uses of UI wage records by State JTPA and other State agencies; 
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• to determine all States' capacity for linking wage-record data with JTPA program data 
based on the characteristics of the two data sets; and 

• to examine State organizational, legal and financial factors that enhance and those that 
limit the uses of wage-record data by JTPA. 

These goals provided a framework for collecting information from all States. A federal/State 
advisory committee (included in the Acknowledgements) was instrumental in designing the 
study. 

A canvass of State JTPA administrators was conducted in April 1991 to document States' current 
data-sharing activities and management information system (MIS) capabilities. A parallel 
canvass of SESA administrators was conducted to determine wage record agencies' ability to 
respond to data sharing requests. Separate canvass instruments were designed to collect the 
needed information from the two agencies. 

The two instruments were sent to the fifty States, the District of Columbia, Puerto Rico, and the 
Virgin Islands. Thirty-six States responded to both the JTPA and wage record instruments; six, 
to the JTPA instrument alone; and eleven, to the wage record instrument alone. Thus, forty-two 
States (representing approximately 96% of JTPA's Title II-A funds in PY 1991 ) responded to the 
JTPA canvass, and forty-seven States responded to the wage record canvass, In addition, twenty 
States submitted samples of data sharing agreements. These exceptional response rates by the 
States provided a firm basis for the study. Appendix II. A contains a state by state list of the 
respondents. 

This chapter presents the findings f/u.n the canvass. It is organized as follows. The wage-record 
reporting system is described in the next section. Then, the States' data sharing experiences are 
documented, and the issue of confidentiality is discussed. This is followed by an examination 
of several other key issues involved in data sharing, such as costs and data accuracy. In the last 
section of the chapter, NGA's conclusions and recommendations are presented. 



Description Of Wage Record Reporting System 

STATE ADMINISTRATION 

Under State Unemployment Insurance (UI) statutes, the State Employment Security Agencies 
(SESAs) have flexibility in how they administer the UI program. While many State laws are 
similar, having been adapted from model language, key differences exist in the types of 
employers, employees and wages/benefits that are "covered" for UI purposes. Coverage 
provisions define the groups of employers who are liable to pay State UI taxes, the groups of 
employees who can accrue earnings to qualify for UI benefits, and the types of earnings that are 
allowable for accrual. Agricultural and domestic service employers are examples of groups for 
whom coverage policies differ across States. Independent contractors, the self employed, and 
military and federal government employees also have different coverage provisions across the 
States. 

To administer the State Unemployment Insurance program, SESAs collect quarterly tax and 
wage information from employers covered in their State. The data collection is funded from 
specific Congressional appropriations for use in administering the UI program. The federal 
government (Internal Revenue Service) collects tax and wage information directly from 
employers for federal UI tax purposes under the Federal Unemployment Tax Act (FUTA) of 
1937. 
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AH States require covered employers to report total quarterly covered earnings within a specified 
period of time. Beyond that, the States have flexibility in how they document each employee's 
earnings and how they establish a reporting system. Differences in the reporting systems across 
States include the selection of data elements to be collected, data collection techniques, 
requirements for data retention, establishment and application of confidentiality provisions to 
release and share data, the data processing environment, and the general administration of the 
data. 

For example, SESAs have theoption to document employees' earnings through "wage reporting" 
or "wage request" procedures in order to establish monetary entitlement to unemployment 
insurance and payment of unemployment benefits. The wage request process requires 
employers to submit wi ge information on a case-by-case basis when it is necessary to process 
an unemployment insurance claim. Wage reporting requires employers to submit all 
employees' earnings data quarterly, whether or not a claim has been filed. 

The Deficit Reduction Act of 1984 required the vast majority of employers in all States to submit 
quarterly information on employee earnings. As mentioned earlier, this was done primarily for 
purposes of detecting fraud in needs-based federal programs. In most States the DEFRA 
requirement was met by the wage reporting procedures described above. All States had a wage 
reporting system by 1988. In Michigan, while the SESA maintains a wage file for fraud detection 
purposes as required under the Tederal Income and Eligibility Verification System (IEVS), the 
wage request process continues to be used to administer the UI program. 

In New York and Massachusetts, the Ul program continues to use the wage request process for 
Ul eligibility and claims determination. In New York State, the wage reporting system is 
administered by the Department of Taxation and Finance; in Massachusetts, by the Revenue 
Department. Michigan and Massachusetts have been considering a conversion to the wage 
reporting process for UI administration, but it is unclear as to when, if at all, this would occur. 

The administrative arrangement concerning the agency responsible for collection and 
maintenance of the wage records in and of itself appears to have little impact on data sharing 
activities. All States and territories except Puerto Rico responding to the NCA wage record 
canvass, including Michigan and New York, currently share wage-record data with at least one 
other entity. For example, in New York State, in addition to the Internal Revenue Service, 
information is released to the Departments of Social Services and Labor under data-sharing 
agreements. In turn, these two agencies have agreements with authorized users. 

The entity legally responsible for entering into data sharing agreements is the SESA 
administrative office, the legal office and the UI administrative office in twenty-one, seventeen 
and twenty States respectively. In four States the Labor Market Information (LMI) agency is 
responsible, and in twelve States a disclosure unit is responsible. 

In thirty-one States the SESA data processing unit, and in thirteen States the UI agency, is 
responsible for conducting computerized data matches. Four States conduct computer matches 
in a central data processing facility, and in one State the LMI unit performs the match. 

The wage record agenciesspent an average of 2.3% of their total computer processing time (CPU) 
to conduct data matching requests. The least time spent on data matching was .($%, and the 
most was 10%. Several SESAs reported thii they were operating computers at their maximum 
capacity or close to it. The processing of increased number cf claims during the current recession 
is not the major reason for mis high utilization rate. Rather, the strains on the agencies' computer 
resources reflect their growing administrative workloads, coupled with continuing budget 
limitations, that have precluded or postponed the upgrading of computer technology necessary 
to meet current processing needs. 




> 1 u 



56 



WAGE-RECORD DATA CONTENT 



In each State the wage record system, and the information collected, reflect the administrative 
needs of the UI agency as reflected in the provisions of the agency's statute. Although State 
wage record systems differ in the data items collected from employers, the following are 
generally collected and maintained: individual employer information (name, ownership, 
address, federal identification number, State UI account number, total monthly employment, 
total quarterly wages and tax contributions) and individual employee information (name, social 
security number and total quarterly wages). 

In the three wage request States, employer identification information, total monthly 
employment, total wages, total taxable wages and tax contributions are collected for UI 
administrative purposes. In these States, individual employee wage information is collected by 
the UI agency only when it is necessary to process an unemployment insurance claim 

All forty-seven States responding to the NGA canvass of SESA Administrators indicated that 
they collect the covered employee's social security number and their total quarterly wages. In 
Michigan and New York, the wage record system contains the same two minimum data 
elements. (Massachusetts did not respond to the canvass.) Table 1 contains a detailed 
breakdown of the information reported under each system. 



TABLE 1 

DATA ELEMENTS AVAILABLE FROM WAGE RECORDS 



Data Elements 


Collected* 


Typically Shared** 


(# of States) 


(# of States) 


Employee Name 


41 


28 


Social Security Number 


47 


29 


Quarterly Wages 


47 


39 


Number of Hours Worked 


1 


1 


Number of Weeks Worked 


7 


5 


Employee Gender 


8 


7 


Legal Name of Business 


44 


29 


"Doing Business As" Name 


44 


21 


Business Address 


42 


27 


Work Location 


24 


11 


State ID Number 


41 


13 


Federal ID Number 


42 


14 


Employer SIC Code 


38 


20 


Other Data Elements 


10 


5 



* Total number of responding Stales is 47. 
** Shared vith other agencies within the State and other requesting entities. 



Employer identification information is the next most commonly collected element. Forty-four 
States collect the name of the business, including the legal and the * doing-business-as" names. 
Forty-two States collect the address of the business. Michigan and New York collect these items 

as well. 

Of the forty-seven States responding to the wage record canvass, only one State collects hours 
worked, and seven States collect weeks workvd. 4 Employee gender is collected in eight States. 5 
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Thus, for purposes of using wage records for JTPA performance standards, the employee's 
identifier (social security number) and their total quarterly wages are the two data items 
maintained by all responding States. The wage-record data would indicate whether or not JTPA 
terminees were employed in covered jobs during a quarter and the amount of their quarterly 
covered wages. If additional data on JTPA terminees were required, such as the industry of the 
employer or other descriptive items, different data files would have to be accessed and different 
procedures would be needed to acquire the data. 

State Data Sharing Experiences 

DATA SHARING IN HUMAN RESOURCE AGENCIES 

With various agencies' competing needs to reduce data collection costs and paperwork buru -n 
and also have a timely and accurate database, administrative data sharing has been increasingly 
viewed as an alternative methodology to post-program data collection directly from participants 
through the telephone survey approach. 

There are a number of questions concerning data sharing agreements. (Two illustrative 
agreements are provided in Appendix II.B, and potential provisions are listed in Table 5 below.) 
What is the capacity of all States to enter into data sharing agreements? Are the States with 
agreements for JTPA's use of wage records more advanced or in a better position to share data 
than other States? More specifically, which agency within a State is empowered to enter Into 
agreements? What capabilities are necessary for each entity participating in the agreement? 
What barriers limit access to the Ul data? 

Because of the confidential nature of wage records, their release typically involves an agreement 
between the data collection agency and the receiving entity. The language in the agreements 
varies from state to state, but generally includes some basic provisions concerning what data 
will be shared, for what purposes, and with whom. (The next chapter of this report provides 
documentation and discussion of the various State laws covering confidentiality along with 
administrative documents guiding their implementation.) For the most part, the agreements 
assure that the integrity of the information will be maintained and that individually identifiable 
data will not be redisclosed. 

Efforts to exchange data have been undertaken for a number of years and under a variety of 
conditions. In recent years, with increasing emphasis on accountability in federal programs, the 
number of data sharing arrangements has been increasing. The NG A canvass indicates that the 
number of agreements has increased by 50% or more in thirty-three States during the past five 
years. No responding State reported a decline in the number of agreements. When asked if a 
25% increase in the number of data sharing agreements cc Jdbcmet, thirty-six States answered 
yes if sufficient funds were provided. Eight States indicated they could not meet such an 
increase. 

The number of data sharing agreements for the responding States was seventeen on average. 
As discussed below, the range of agreements per state is from one to 104 agreements. Data 
sharing agreements have been made with all levels of government, other States, universities, 
research entities and with private sector entities. For example, thirty-six wage record agencies 
responding to the canvass have agreements with a federal agency. 

Overall, because of federal requirements, forty-three wage record agencies have a data sharing 
agreement with the Child Support Enforcement program. The high volume of data sharing 
relates to the mandatory nature of this activity. For employment and training programs, 
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wage-record data are accessed in twenty-eight States by AF DC /JOBS, in twenty-six States by 
Food Sin-ops, in nine States by post-secondary vocational education, and in twenty-nine States 
by 'TPA. The AFDC/JOK5 and Food Stamps programs have requirements to use wage-record 
data, while the other two programs do not. 

In most cases an office within the SESA, typically the administrative or the legal office, is 
responsible for entering into data sharing agreements. The UI office is also legally responsible 
for entering into agreements. 

Twenty-one out of forty-two wage record agencies indicated that they use standard language 
in all data sharing agreements. In a review of the data sharing agreements provided by twenty 
States, the language, while having some common characteristics, appears to be individualized 
by State. In some agreements, such as those with the Child Support Enforcement Agency, the 
language is generally consistent across States because of federal requirements. 

Although the type of agreement and the language therein appear to reflect State differences in 
the way business is done, there do not seem to be any real barriers to data sharing that are 
imposed by the agreement itself- It appears from the review of the existing agreements, mat one 
or several model agreements could be drafted for States to use as a yardstick to compare the 
completeness and adequacy of their documents. 

JTPA/WAGE-RECORD DATA SHARING 

Wage-record data can be used for multiple purposes by a variety of agencies. For example, they 
can be used to verify program eligibility, to examine the employment and earnings patterns of 
individuals before and after participation in the program, and to evaluate the program and /or 
to establish performance standards. 

Of the States responding to the NGA canvass, twenty-nine reported data sharing agreements 
between UI and JTPA State offices which link individual JTPA records to wage records. Of these 
twenty-nine States, three use the wage record information only to verify eligibility under the 
Economic Dislocation and Worker Adjustment Assistance (EDWAA) Act of 1988, and one uses 
the information only to validate the JTPA 13th-week post-program telephone survey results. 
The other twenty-five States use the wage-record data for program accountability purposes. In 
addition to the twenty-nine States, another two States regularly receive lists of UI claimants who 
are potential JTPA Title II-A or EDWAA participants, and one State exchanges data on a 
project-by-project basis. 

Nineteen of the Ctates with data sharing agreements art participating in NCEFs study of the 
feasibility of using wage records for accountability purposes. This large number, relative to the 
total number of States with data sharing agreements, suggests that the NCEP project has been 
instrumental in encouraging States to forge JTPA/UI data sharing agreements. 

Officials in eight of the seventeen States with no data sharing agreements reported that the JTPA 
office has not requested such agreements. In three other States, the JTPA office had sought but 
was not able to forge an agreement with the wage record office. Confidentiality barriers and 
costs were the reasons given for the absence of data sharing agreements. 

State officials reported an average annual match rate of 83% between the wage records and JTPA 
participant records, with a range of 70% to 93%. Approximately 60% of the States have match 
rates above 85%. (The average annual match rate is the number of JTPA participants for which 
UI wage information is available as a percentage of the total number of JTPA participants 
submitted to the UI office for computer matching purposes.) 




Hie data sharing agreements between the wage record and JTPA offices are primarily for JTP A 
Title II-A and EDWAA programs. Most of the wage-record sharing agreements do not include 
the 3-percent set-aside for older workers or the 8-percent set-aside for coordination with 
education. 

One might expect that data sharing agreements would occur more frequently when the JTPA 
and wage record office are in the same department rather than separate departments. However, 
the canvass found that data sharing agreements are as likely to occur in States where the two 
offices are in separate departments as in States where the two offices are in the same department. 

Data sharing agreements are more likely to exist in States that maintain their JTPA databases on 
mainframe computers as opposed to smaller computers. This may be due to an increased 
tendency for formal agreements to be required when a centralized data processing unit is 
responsible for the JTPA and UI data. Another explanation is that data sharing is more likely 
to occur when the JTPA and UJ records are resident on a common mainframe computer. 

As might be expected, State JTPA agencies with UI agreements to access wage records also have 
more data exchange agreements with other human resource agencies than State JTPA agencies 
without UI agreements. For example, 67% of the JTPA agencies with UI agreements also have 
data exchange agreements with the welfare employment program as compared to 44% for JTPA 
agencies without UI agreements. The same tendency holds with regard to employment service 
and vocational education programs; see Table 2. 



TABLE 2 

JTPA AGREEMENTS WITH OTHER EMPLOYMENT 
AND TRAINING AGENCIES 





States With 
JTPA/UI Agreements* 


States Without 
JTPA/UI Agreements 


AFDC/IOBS 


67% 


44% 


Employment Service 


53% 


26% 


Vocational Education 


27% 


7% 



* Total number of responding States is 42. 



Officials in States with data sharing agreements were asked to describe any problems the JTPA 
agency has encountered with the data received from the wage record office. In six States, 
problems have emerged. The main problem has been the time lag. Another State reported 
technical difficulties at the beginning of the data sharing arrangement, although this problem 
was quickly resolved. 

CONFIDENTIALITY OF WAGE RECORDS 

No single issue has evoked the concern of data users and producers and those within the 
statistical community more than confidentiality. The challenge is to ensure the protection of the 
data through confidentiality provisions while also allowing reasonable access to the data 
through data sharing agreements. This issue is being brought to the fore by several factors: 
advances in computer technology; administrative strains caused by shrinking budgets, 
paperwork limits, program accountability requirements and demands for inter-agency 
coordination; and changes in the locus of decisionmaking which can complicate data 
management. 
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There are two major questions about data confidentiality that affect data sharing within JTPA. 
First how do differences in a State's confidentiality provisions affect its ability to implement 
data sharing arrangements? Second, how do differences in States' confidentiality provisions 
affect VS. DOL's ability to access State wage data for national JTPA performance standards? 
These questions concerning the extent of access allc vable under confidentiality provisions raise 
important issues about the types of actions States would need to take in order to establish 
agreements for data sharing, both within and between States. 6 

The available literature and ongoing discussions of the "confidentiality versus access" issue 
generally examine legal, technical, financial, intent/purpose of usage and misuse, informed 
consent usage a..d methods, and the perceptions of both data producers and users. Some of 
these issues are addressed in the next chapter of this report. The present chapter approaches 
data access from an accountability perspective, i.e., the conduct of data sharing activities in a 
controlled environment. 

Thomas Plewes of the Bureau of Labor Statistics (BLS) has pointed out the need for the 
"development of a unified, sound and meaningful approach to controlling the sharing of date 
covered by a pledge of confidentiality with other entities. A "controlled sharing" approach is 
not completely in place yet, and the question of how to accomplish it remains to be fully worked 
out. 

To frame confidentiality as an accountability issue, an analogy can be made between data and 
other valuable public sector resources. Just as personnel, fiscal and other institutional resources 
are managed for accountability and other purposes, so should data resources. The government 
should become more accountable for data resources just as it is becoming more accountable for 
the expenditure of funds and the resulting outcomes. 

Control over the uses of wage-record data is necessary because there is public concern over both 
the amount of data collected by the government and the data's ultimate uses. Examples that 
suggest the public' s reactions to government date activities include: 

• low and uneven response rates to the 1990 Census; 

• the support of employers, particularly small businesses, for the Paperwork Reduction 
Act of 1991 (S. 1139); and 

• Slate legislative responses to misuses of personal data (Caller ID; private sector use 
and selling of information-EQUlFAX; Department of Motor Vehicles' releases of data 
that had some unfortunate results). 

The above examples suggest the public may perceive the "controlled part" to be less than 
adequate and that the fiduciary responsibility of the government is not being upheld. This 
argues the case for improved management and accountability of data sharing. However, while 
these examples imply individuals and businesses are concerned about the government's role in 
collecting and using data after its receipt, there is no solid proof that controlled information 
sharing allowed under consensual arrangement would sacrifice respondent cooperation and 
data validity. 

Despite the legitimate concern over confidentiality, forty-six States responding to the NGA 
canvass are sharing wage-record data. (Puerto Rico is the exception.) The number and nature 
of data sharing vary considerably among the States: from one that has just a single agreement 
to another with 104 agreements. The average number of data sharing agreements per state is 
seventeen. Table 3 presents the frequencies associated with given numbers of agreements. 
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TABLE 3 
NUMBER OF AGREEMENTS 



Number of Agreements 


Number or states' 


0-4 


9 


5-9 


15 


10-19 


10 


20-29 


6 


30-39 


1 


4049 


1 


50+ 


1 


n/a 




• Total number of responding Stales is 46, 

"These three States did not respond to the Ul-side instrument but reported agreements on the JTPA-SMte instrument 



Data access is currently governed by a myriad of federal and Slate laws and regulations 
regarding privacy or confidentiality and open access or open records laws. In forty-one States, 
the State Ul statute affects the release of wage-record data. In the States where the wage record 
is the responsibility of an entity other than the SESA, other laws govern the release of the data. 
Twenty-three States reported that confidentiality laws affect data release. In ten States, freedom 
of information laws also have an impact. Table 4 summarizes this information. 

At the federal level, the VS. Bureau of Labor Statistics' confidentiality rules govern the release 
of Standard Industrial Classification (SIC) data and other information for worksites of multi-unit 
firms. The U.S. DOL's proposed regulations, published for comment on March 23, 1992 (57 FR 
10064), stipulate the following: aside from mandatory disclosures in the Social Security Act, the 
Federal Unemployment Tax Act, and the Wagner Peyser Act Section (303XaXl) of the Social 
Security Act governs the confidentiality and disclosure of data collected and maintained for the 
administration of the unemployment insurance (Ul) program. 



TABLE 4 

LEGAL STATUTES AFFECTING RELEASE OF WAGE-RECORD DATA 



Legal Statutes 


Number of States* 


State Ul Laws 


41 


Privacy/Confidentiality Laws 


23 


Freedom of Information Laws 


10 


Other- 


6 


* Number of responding States is 47. 



Reviews of the confidentiality provisions in data sharing agreements provided by twenty States 
reveal that the language ranges from very broad statements that the date are confidential in 
general, to very detailed provisions. The most common provisions are: the purpose for date 
sharing, who has access, who pays, and data usage. Provisions regarding date retention and 
final disposition were the least likely to be incorporated into dita charing agreements. Table 5 
presents the number of States including each provision in their data sharing agreements. 
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The agreements of several States seemed to be very similar, suggesting there was model 
language and/ or interstate communication when the agreements were drafted. Several of the 
agreements contain the major elements and offer possible models for other States to emulate. 
Appendix II.B contains two candidates. 



TABLE 5 

CONFIDENTIALITY PROVISIONS IN DATA SHARING AGREEMENTS 



Confidentiality Provisions 


Number of States* 


Allowable Use of Data 


19 


Cost Reimbursement 


18 


Who Receives Data 


18 


What Information is to be Shared 


17 


Redisciosure of Data 


16 


Security Procedures 


14 


Liability Clause 


11 


Enforcement 


6 


Informed Consent 


5 


Data Retention 


3 



* Total number of responding States is 20 



Of the States with JTPA data sharing agreements, the most common provision concerning 
confidentiality was one specifying the types of allowable data uses. Six States reported that 
provisions covered release to local entities, and six States had provisions for release to 
out-of-State entities. (See Table 6.) 



TABLE 6 

CONFIDENTIALITY PROVISIONS IN JTPA DATA SHARING AGREEMENTS 



Confidentiality Provisions 


Number of States* 


Release to 




Local Entities 


6 


Out-of-State Entities 


6 


Federal Agencies 


3 


Private Entities 


2 


Types of Allowable Data Use 


8 


Data Retention Limits 


4 


Other 


6 


* Total number of responding States is 29. 



Concern over confidentiality has led to limits on data access: some requesting entities have not 
received access to all the data they would like to use. Twenty-nine States cited confidentiality 
as a reason for not being able to meet mn-JTPA data sharing requests, and five Steles cited it as 
a reason for not being able to meet JTPA requests. Whether confidentiality was *he sole reason 
for the lack of an agreement is difficult to say in the absence of further information about the 
purpose and nature of the requests that were not met. 
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Many SESA Labor Market Information Offices maintain records of the volume and types of date 
releases to users. This information is useful in justifying data collection and in addressing the 
public responsibility to disseminate data for legitimate purposes. Agencies' records of release 
of confidential data to other entities through data sharing agreements need to be clear and 
readily understandable. 

At the national level there are several initiatives that could improve the accountability of data 
sharing activities. The VS. DOL is attempting to standardize the Ul data sharing process, 
including confidentiality provisions, by specifying who may receive what data under what 
conditions. The VS. DOL's proposed regulations reinforce the basic concept that confidentiality 
of State UI records is essential. Under the proposed regulations, individually identifiable data 
could only be shared with a non-Ul entity under a written agreement that specifies the: 

• purposes; 

• safeguards; 

• reimbursement of costs; 

• method of disposal of data after the agreement terminates; 

• approved uses; 

• conditions of any redisclosure; 

• inspection and oversight responsibility of the UI agency; and 

• the use of informed consent procedures for claimants and employers. 
Other federal initiatives include the following: 

• The Bureau of Labor Statistics is especially concerned with the unrestricted use of 
employer data and is designing an approach to confidentiality for the Business 
Establishment List program. 

• The Office of Management and Budget is drafting an administration bill for federal 
interagency sharing of date under certain procedures and safeguards but will most 
likely not offer provisions concerning the States. 

• The National Academy of Sciences has been conducting a study over the past 18 
months to examine issues of privacy protection, facilitating responsible data access, 
and enhancing the public's confidence in government data. This effort is supported 
by the National Science Foundation, Census Bureau, Bureau of Labor Statistics, 
Internal Revenue Service, National Institute on Aging, and National Center for 
Education Statistics, among others. 

The proposed regulations on the confidentiality of Ul data, in particular, and the policies of BLS 
indicate a trend toward standardizing the data sharing process. Such standardization will 
necessarily affect SESA date sharing practices. In addition, States may decide to take action 
beyond these national measures. 

In summary, many States have dealt successfully with the legal issues, at least in the sense that 
they have entered into functional data sharing agreements which address many of the issues 
outlined above. Establishing an audit trail for confidential UI data and their release is a further 
step toward sound and prudent management of these data resources. Many SESAs will 



welcome the proposed regulations on confidentiality and the opportunity to conduct data 
sharing in a more controlled environment; others may think the regulations are limiting. 

Key Issues In JTPA/Wage-Record Data Sharing 

COSTS OF DATA EXCHANGE 

The costs of developing and maintaining a system to link wage records and JTPA administrative 
records have been found to be relatively low. An estimate of the start-up costs is approximately 
$20,000 per state for the JTPA office. 9 State officials responding to the SESA canvass estimate 
that it will cost the wage record office an average of $8,000 to develop a system to respond to 
JTPA data requests. (These start-up costs for the State wage record offices ranged from $600 in 
a small State to $20,000 in a large State.) These costs are required to be reimbursed to the wage 
record office by the State JTPA agency. 

Annual operating costs of such a system are low and vary according to the size of the State, 
based on information obtained from the canvass. The average annual cost per record for the 
State JTPA office to prepare the participant data for matching and processing the Ul wage data 
once obtained is $1.75, with a range across States of approximately $1.00 to $230. On 

average, 

it costs the wage record office $020 per requested record to process and match the JTPA records. 
Thus, an estimate of the total average cost per record for the system is $1.95, 10 As discussed 
earlier, the U.S. DOL has proposed regulations affecting the confidentiality and oversight of Ul 
data. If these regulations are implemented as proposed, there likely would be an increase in the 
cost of obtaining data from the State Ul offices, due to additional oversight responsibilities that 
would be required. 

If a JTPA /wage-record data exchange system had been in place during PY 1989, it would have 
cost a little over $700,000 to obtain the data for the 362,713 Title II-A adult terminees nationwide 
and almost $460,000 for the 234,843 EDWAA terminees. Using the estimated average cost per 
record given above, it would have cost the State with the largest number of JTPA participants 
around $65,000 to obtain wage record information for all its Title 1I-A adult terminees and 
$13,800 for its EDWAA terminees. (These figures do not take into consideration possible savings 
produced by returns to scale.) It would have cost the State with the smallest number of 
participants approximately $1300 for its Title II-A adult terminees and $600 for the EDWAA 
terminees. 

A main component of the present JTPA performance standards system is a post-program 
telephone survey, as noted at the outset of this chapter. According to a study by the National 
Governors' Association, the average cost for conducting JTPA telephone follow-up in PY 1989 
was $19 per completed interview. Following assumptions developed by Northern Illinois 
University regarding numbers of participants, sample sizes and respondent rates, 11 but 
modifying the cost per completed interview to reflect current figures; the annual cost of 
collecting JTPA post-program data is estimated to be $3,068,500 ($2,141,300 for Title II-A adults 
and $927,200 for Title III). This current cost is substantially more than the $1,160,000 projected 
cost of obtaining earnings and employment information through the use of Ul wage records. 

Who saves from a switch from the post-program telephone survey to wage-record data is 
determined by who conducts the present survey and who pays for it. According to the canvass 
results, two-thirds of the States conduct centralized JTPA follow-up. This proportion is the same 
for both States with JTPA/ wage-record data exchange agreements and States with no 
agreements. Of course, actual savings to States and /or Service Delivery Areas (SDAs) would 
also be determined by the type of agreement between the State JTPA office and SDAs to finance 
the post-program data collection. (For further discussion, see Appendix LB.) 
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The canvass produced no evidence that the cost of maintaining a JTPA / wage-record data 
exchange system varies with the type of State management information system. The costs seenri 
to be similar for States with centralized automated systems and for States with decentralized 
automated systems maintained at the SDA level. At this time, all States have automated MIS 
systems. 

The type of hardware (e.g., mainframe vs. micro computer) used to maintain the JTPA data also 
seems to have no effect on the cost of the JTPA/UI exchange system. However, as noted before, 
more States with JTPA/UI data exchange agreements maintain JTPA data in mainframe 
computers than States with no agreements. Over the last five years, computer hardware and 
software have improved greatly. It is possible currently, as many States do, to maintain large 
databases in small personal computers. In fact, in three States changes are under way to transfer 
the JTPA databases from mainframe computers to personal computers. 

A key issue regarding data exchange costs in a few States is the need to develop a flow of 
individual participant data between the SDA level and the State level. Five JTPA State 
management information systems (Alabama, Hawaii, New Mexico, New York, and Puerto Rico) 
do not contain individual participant data. In three of these States, it would take more than five 
weeks to obtain JTPA individual records from the SDAs; in the other two, it would take between 
one and four weeks. 

TIME LAGS AND DATA AVAILABILITY 

The wage-record data need to be available on a timely basis for purposes of JTPA program 
management and performance standards. Availability refers to both which data are available 
(Le., the data items collected and retained) and when these data are available (i.e., the time 
required to process and prepare the raw data for internal agency administration and record 
matching uses). 

A major concern regarding the use of wage records for determining JTPA performance 
standards is the time lag between the end of a calendar quarter, which is the reporting period 
for the data describing individuals' wages, and the date when the data become available for 
record matching. This depends on the timing of the receipt of the raw data from employers and 
the time required for data entry and processing. 

Covered employers are required to submit a wage and tax report to the Ul agency within one 
month of the end of each calendar quarter. If the information is not received, the employer is 
considered to be delinquent and is subject to tax penalties. Upon receipt of the forms submitted 
by employers, data are entered into agency computer systems. The time it takes to enter and 
process the data contributes to the overall time lag. 

SESA officials report that on average it takes twelve weeks between the end of a quarter and the 
availability of the wage-record data for matching purposes. Although the number of weeks 
ranges from four to twenty-six, there is little variation among the vast majority of the States. 
(See Table 7.) In forty-three States, the information is available in less than fifteen weeks. Only 
in three States does it take more than fifteen weeks for the wage-record data to be available. 
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TABLE 7 

TIME LAG FOR WAGE DATA AVAILABILITY 



Number of Weeks 


Number of States* 


0-9 


7 


10-14 


36 


15-19 


1 


20 + 


2 


* Total number of responding States is 46; one State was not abl 


e to provide this information. 



In the three States where it takes more than fifteen weeks for the wage-record data to be available, 
the main reasons are data submission media and data entry. The media used by employers fall 
into two broad categories: (1) hard copy and (2) various electronic media including magnetic 
tapes, floppy disks, and interactive input. Overall for the SESAs responding to the canvass, 
83.5% of employers wilh fifty or more employees, and 89% of employers with less than fifty 
employees, submit their wage data on hard copy. Filling out the hard copy forms takes 
considerable time, and the information then has to be entered into the State's computer system 
by State agency staff. 

The turnaround time between JTPA officials' request for wage record information and their 
receipt of the requested information adds to the amount of time it takes wage-record data to 
become available. State JTPA officials were asked to estimate the turnaround time from the date 
they submit a request for wage record information to the date they receive the information. On 
average, it takes three weeks for the wage data to be available to the State JTPA office. The lag 
time ranges from one to six weeks. It should be noted that some JTPA offices, in States where 
JTPA and wage record offices are under the same agency, have immediate access to wage 
records. 

The time lag between the date of request by the JTPA agency and the receipt of wage-record 
data seems to be affected by the volume of the request. For requests of 10,000 records or more, 
the average number of weeks from the time the request is submitted to the time the information 
is received at the State JTPA office is 3.25 weeks; for requests of less than 10,000 records, the 
amount of time is only 2.00 weeks. 

Another issue regarding the use of wage-record data for JTPA purposes concerns the availability 
of the specific data items requested. Availability refers to both the capture of the specific data 
on the original collection instrument and the length of time these data are retained, either in 
active files or archives within the State. Fourteen States reported problems with non-JTPA 
requests because the requested data were not available. Also, two JTPA officials reported that 
their request for wage-record data could not be met because the requested data for pre-program 
quarters had not been archived. 

States generally retain wage-record data (name, social security number and quarterly wages) in 
active files for an average of eight quarters. However, there are variations in retention across 
data elements depending in part on the way in which State files are organized. The employer's 
Standard Industrial Classification (SIC) code, which is stored separately from the wage data in 
some States, is retained on average for thirteen quarters. 

Once archived, the name, social security number and quarterly wages are retained in storage 
for an average of six, five and five years respectively. The SIC code is retained in storage for six 
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years on average. Some data elements, such as the individual employee's name, are retained 
indefinitely. 

Because the most commonly requested wage-record elements are retained for a considerable 
amount of time after collection, States are usually capable of meeting legitimate requests. On 
the whole, data availability should not cause significant problems within the States or at the 
national level. 

Different uses of wage-record data by JTPA offices make different demands on timeliness. For 
example, if wage records are used for studies of pre- and post-program earning patterns, or for 
net-impact evaluations, timeliness is less crucial than if the wage records are used for 
performance standards. Timeliness is critical to performance standards because the data are 
required immediately after the end of the JTPA program year for purposes of determining 
incentive grants. 

In summary, for the typical State responding to the canvass, it will take about fifteen weeks 
between the end of the reporting quarter and the time when the wage records are available to 
the JTPA office. In the three States with a time lag beyond fifteen weeks, it would be difficult 
for them to use wage-record data for JTPA performance standards purposes, as their systems 
are currently organized. 

OUT-OF-STATE EMPLOYMENT 

Another critical issue regarding the use of wage records for JTPA accountability purposes is 
out-of-State employment. When JTPA participants are placed in jobs outside the State where 
they were trained, the program's effect could go unmeasured. This situation would most likely 
arise in Service Delivery Areas bordering other Stares. The overall effect for the JTPA system 
seems to be small, but the exceptions warrant attention. (See also the discussion in Chapter IV.) 

Based on nationwide data from the U.S. Bureau of the Census, there are twenty-one Metropolitan 
Statistical Areas (MSAs) that cover mere than one State. The map on the following page shows 
the locations, and Appendix H.C gives a complete list of the MSAs. A total of twenty-seven 
States have MSAs that are in multiple States. Of these twenty-seven States, twenty-four 
responded to the JTPA canvass, and eleven reported data exchange agreements with 
neighboring States. 

State JTPA officials were asked to estimate the percentage of JTPA terminees residing in their 
State but placed in jobs outside the State's boundaries. The estimated percentage of out-of-State 
employment averaged 3.5%. State officials were also asked to list SDAs where this number 
represents at least 10% of the total number of JTPA terminees. None reported any such SDAs, 
although some respondents recognized out-of-State employment as a serious issue for some of 
their SDAs. This issue seems to be particularly import" nt in the New York City and Washington, 
D.C. areas. 

While the number of SDAs seriously affected by this problem seems to be few, out-of-State 
employment has to be addressed in these special cases to assure equitable treatment throughout 
the JTPA system. Since the data would be used to determine performance and to distribute 
incentive dollars for exceeding standards, affected SDAs would want interstate data sharing 
agreements or at least a statistical adjustment mechanism to account for out-of-State 
employment. 
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DATA CONTENT AND APPLICABILITY 



The use of UI wage-record data for JTPA accountability purposes appears to be technically and 
legally feasible for a majority of States, as demonstrated by the results of this study and others. 
A next issue relates to how fully these date meet the needs of local, state and national 
decisionmakers responsible for the various aspects of JTPA program management, monitoring 
and evaluation. Previous assessments of wage record content generally have focused on the 
scope of coverage of employers in the wage record system and differences in coverage across 
States. 13 The benefits of the information provided by each source depend on the application: 
state/local program management, national performance standards and/or evaluation. The 
content of the wage-record data for each of these applications will now be discussed. 

For current performance standards purposes, the UI wage records and the follow-up survey 
provide comparable information. The quarterly UI wage data, whim attached to the JTPA 
participants' characteristic data, provide basic information about post-program employment 
and earnings, as do the corresponding survey data. However, it should be understood that the 
specific measures of performance derived from tin? two data sources are not directly comparable* 

For other program management purposes, there are significant differences between the content 
of the data collected from the current telephone survey and the UI wage records. The 
post-program survey requests information on whether the respondent worked for pay during 
the thirteenth week after termination; the amount of hours worked that week; the hourly wage 
rate; the extra compensation received for overtime, bonuses, tips, etc.; ami the total number of 
weeks employed during the thirteen-week period, In addition, State JTPA agencies may include 
other questions on the survey instrument. 

In contrast, the States' wage-record data indicate whether an individual was employed or not 
during the quarter and the total quarterly wages reported. In a majority of the States, there is 
no information on the wage record that would indicate the date the person began employment 
or the duration of employment during the quarter. Thus, for purposes of local program 
management, a participant survey can provide a more detailed description of the post-program 
employment experiences of JTPA participants at a particular point in time. 

The eight States t ha t collect informa lion on hours (or weeks) worked have the ad vantage of being 
able to use a more precise measure of employment and to infer an hourly (or weekly) wage rate. 
The potential for other States to collect these items through their wage reporting systems seems 
limited at this time. 

During the 1980s, several States found that their wage-record data were insufficient to meet all 
of their goals for program management, since an hourly (or weekly) wage rate could not be 
obtained, without collecting the hours (or weeks) worked during the quarter. 14 In addition, 
without knowing the individual's occupation after leaving the program, the relationship of a 
post-program job to the type of training could not be determined. 

These agencies examined the feasibility of adding the needed data elements to the wage 
reporting form. The additional items generally considered for inclusion on the form were hours 
{or weeks) worked by employee and an occupational identifier. The States undertaking these 
efforts found the task of augmenting the wage reporting: form to be difficult as best. However 
in at least one circumstance, the attempt was successful. 5 

Occupational information is also not typically collected on the current survey form. However, 
if JTPA program managers would like to obtain this information, it would be easier to use a 
survey approach than to add additional data items to th f ^ UI wage reporting system. 
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As stated earlier, the telephone survey and Ul wage data are comparable for performance 
standards purposes. However, for other accountability purposes, they are not comparable. For 
example, there are variations in the content of Ul wage data across States, industries and 
employers. The reporting of wage data in a payroll data system can be affected by employee 
overtime, bonuses, shift differentials, part-time work, cost-of-living differentials and other 
circumstances. Establishments have different-length work weeks (varying on average from 
thirty-five to forty hours) and have different methods of rompensating for overtime hours 
worked. The usual absence of length-of-time- worked data further restricts interpretation of the 
quarterly wages reported by different employers in the wage record system. Since persons 
completing the same training programs may enter jobs in a variety of industries where 
employment patterns and compensation methods differ, the Ul wage data do not suffice for 
in-depth evaluations of trainees' employment and earnings outcomes. 

On the other hand, for longitudinal analysis and evaluation of job training effects on 
participants' earnings* the Ul data have advantages. The wage records can be used to trade the 
quarterly employment status and earnings of participants for long periods of time before and 
after their receive their training. Since the post-program survey data are only collected once, 
these data cannot be used for longitudinal analysis. Furthermore, VS. DOL's separate Job 
Training Quarterly Survey, a series of national snapshots of participant characteristics and 
immediate outcomes, also lacks longitudinal information about individual participants. 

In addition, the Ul wage-record data could serve as a "common currency" to consistently 
describe earnings outcomes across a variety of public employment and training programs. The 
wage data can be obtained for nearly the universe of program participants (as opposed to just 
a sample when the survey approach is used for programs with many participants). As a 
standardized source of data, the wage-record data could be used to examine issues that crosscut 
the services delivered by various providers and to facilitate a coordinated effort toward common 
employment and training goals. 

In recap, Ul wage records provide a source of basic information on employment status and 
wages of foi mer JTP A participants which is comparable to the follow-up survey data for national 
performance standard uses. They also are useful for longitudinal analysis and evaluation of job 
training effects on participants' earnings. However, these records by themselves do not provide 
the in-depth information needed to address the full range of program accountability and 
evaluation purposes. 

In order to meet all of these information needs, US, DOL should consider a third option in lieu 
of using either wage-record data exclusively or telephone survey data exclusively. It should 
examine the feasibility of using wage-record data together with a restructured participant 
follow-up survey that all States could conduct on a regular basis. 

This alternative approach could include the use of the wage-record data to indicate employment 
status and earnings of program participants and the conduct of a modified follow-up survey to 
gather additional job information not available in the wage-record database. One- possibility is 
to have statewide follow-up surveys, with smaller sample sizes that are calibrated to the 
wage-record data, This would reduce the costs of the follow-up survey and would allow 
additional evaluative information to be integrated with the employment status and earnings 
data obtained from the wage records. This approach could reduce costs and reporting burden, 
yet yield more meaningful data to improve the understanding of training program outcomes. 
At least one State has taken an approach similar to this one. 18 

In conclusion, to address the full range of program accountability and evaluation purposes, the 
U.S. DOL and the States should explore this and other options that capitalize on the benefits of 
the wage-record data when combined with other types of data collection, 
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DATA ACCURACY 



Inaccuracies can occur in both survey data and administrative records. For the most part 
inaccuracies arise from reporting errors made by the original data provider and /or clerical errors 
in transcribing the information. Errors can exist in the data items used to link data from two 
different sources (e.g., social security number) and /or in the data used for program management 
(e.g., participant's post-program earnings). 

With telephone interview data, inaccuracies primarily occur when respondents misreport their 
own employment status and/or earnings. State JTPA offices are required to validate data 
received through the follow-up interview used in the performance standards system, but the 
process is difficult. 

With UI wage-record data, employees' names and social security numbers may be missing or 
may contain invalid or duplicate numbers. Given the hard copy submission format that is atill 
so widely used by employers to report quarterly wage and tax information, large amounts of 
data must be entered into State computers. These data are more likely to contain errors than 
data submitted electronically by employers. 

Since the wage record is not edited fully at the time of data entry, errors may go undetected. An 
individual's wage-record data generally is verified only when that person files a claim for 
unemployment insurance benefits based on the reported wages. On average about 10% of the 
wage records are verified annually. Of the forty-seven wage record offices responding to the 
NGA canvass, twenty-five States verify missing social security numbers, twenty-three verify 
invalid numbers, and sixteen verify duplicate numbers during the UI claim determination 
process. 

The wage record agency generally does not guarantee the accuracy of an individual's 
employment and earnings record and waives the liability resulting from use of these data. 
According to the canvass, eleven States include clauses in data sharing agreements stating that 
the employment and wage data are unverified. In some cases, where the data are used for 
enforcement and legal purposes, the interagency agreement requires the requesting agency to 
independently verify the data. 

Only two States reported any problems with the accuracy of the wage-record data obtained for 
JTPA purposes. This if, encouraging, since inaccurate data could adversely affect the usefulness 
of the data for performance standards purposes. In addition, inaccuracies in the social security 
number used to link the wage-record data to the JTPA participant records could make data 
matching difficult if not impossible. Wage record agencies were not asked in the canvass to 
assess the level of accuracy of data items contained in the wage-record data system (e.g., the 
social security number, quarterly wages, employer identification number, SIC code or the 
business address). However, JTPA officials in States with data sharing agreements were asked 
to indicate how they perceived wage-record data accuracy. Generally, they perceived the data 
to be adequate for their purposes. 

Summary And Recommendations 



This chapter has documented and examined the capacity of States to share UI wage-record data 
with other State agencies, particularly with State JTPA offices. Wage-record data offer an 
appropriate source of data for use in the national JTPA performance standards system because 
(1) the data sharing capability exists, (2) the costs of data matching are low, and (3) the 
wage-record data are comparable in content to the current post-program survey data for 
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performance standards purposes (although the specific measures of performance derived from 
the two data sources are not directly comparable). 

However, before these data can be used by the States for performance standards, several 
practical issues need to be resolved. The recommendations in this section address these issues. 
For example, U.S. DOL will have to modify its performance standards requirements regarding 
the timeframe for calculating standards. Also, some State JTPA management information 
systems do not contain the necessary individual participant data for matching purposes, and 
some wage record agency computers are operating at full capacity and therefore cannot meet 
additional requests without additional resources. Other important issues include 
confidentiality, cost sharing, and data accuracy. In many cases these issues call for State-specific 
technical assistance. 

In addition, to assure equitable treatment of SDAs regarding performance standards and the 
distribution of incentive funds, those SDAs significantly effected by out-of-State employment 
would want their States to develop interstate agreements or at least provide a statistical 
adjustment procedure. These procedures should be developed and tested with the affected 
SDAs. 

PERFORMANCE STANDARDS SYSTEM 

The National Governors' Association makes the following recommendations with n gard to 
using UI wage-record data for JTPA performance standards. 

1. The U.S. Department of Labor should provide States with the option of 
using UI wage-record data in place of telephone survey data for purposes 
of JTPA post-program performance standards. 

There are three primary justifications for this State option, based on the results of this canvass. 
First, efforts to share wage-record data have been undertaken over the years under a variety of 
conditions and for many purposes including research, program accountability, and law 
enforcement. Because of the increased focus on accountability c r human resource and other 
public programs, the number of data sharing agreements, regardless of the agency responsible 
for the wage-record date, has increased dramatically in recent years. At least forty-six 
wage-record agencies each share their data with at least one other entity, and the average 
number of data sharing agreements is seventeen. Given the amount of data sharing already in 
place, it appears that virtually all State wage-record agencies should have the capability to share 
their quarterly wage-record data with requesting State JTPA agencies. 

In addition, a majority of the State JTPA offices already have agreements to access the wage 
records. Specificaty, twenty-nine States reported data sharing agreements between their wage 
record and JTPA offices. Three of these States use the wage record information to verify EDWAA 
eligibility. One State uses wage-record data to validate the JTPA 1 3th- week post-program 
telephone survey results, and twenty-five States use it for accountability purposes. 

Second, the cost of the wage-record data is relatively low. The one-time implementation costs 
average about $28,000 per state ($20,000 for the State JTPA office and $8,000 for the UI office). 
The average annual cost per record for the State JTPA office to prepare the participant data for 
matching and then processing the wage-record data once obtained is $1.75, and the average 
annual cost per record for the UI office to process and match the JTPA records is $050. The 
average cost of $1.95 per record using the UI approach is much lower than the average cost of 
$19 for a completed telephone interview using the current survey approach. However, it should 
be noted that implementation of the proposed VS. DOL regulations on confidentiality and 
oversight of UI data may increase the costs to the UI agency. 
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If the wage record matching procedure had been used in PY 1989, it would have cost about 
$1,160,000 to obtain roughly comparable employment and earnings data on all adult Title Il-A 
and Title 111 terminees. By contrast, it is estimated that the annual cost of collecting JTPA 
post-program information for adult samples using the current telephone survey is $3,068,500. 
This indicates potential cost savings to the JTPA system of about $2 million annually. 

Third, for use in national JTPA performance standards, the employment status and quarterly 
earnings from the wage records are roughly comparable to the employment status and 
13th- week earnings data currently obtained from the post-program telephone survey. In 
addition, the Ui wage information might serve as a common currency to measure the 
perf crmance of JTPA and other employment and training programs. 

2. The U.S. Department of Labor should modify the JTPA performance 
standards system to accommodate the use of UI wage-record data. 

If wage records are to be used for JTPA performance standards purposes, timeliness is essential. 
Currently, data from four quarters (one from the previous Program Year and three from the 
current Program Year) are used immediately after the close of the current Program Year to 
calculate performance measures for the adult programs. Modifications to this timeframe must 
be made because wage-record data from the last quarter of the current Program Year would not 
be available in time to determine SDA performance for purposes of incentive grants. 

SESAs reported it takes an nverage of twelve weeks for wage-record data to be prepared and 
made available for matching by JTPA offices, with a range of four to »wenty-six weeks. In three 
States, SESA officials reported that it takes between seventeen and twenty-six weeks from the 
end of a quarter until wage-record data are available to the JTPA office for matching purposes. 
Under current procedures these three States would not be able to obtain the wage-record data 
on time for determining performance standards. 

Several options exist to address the time lag problem. One option would be to use wage-record 
data from the four quarters of the previous Program Year and none from the current Program 
Year. Another option would be to use two quarters from the previous Program Year and two 
quarters from the current Program Year. The latter option appears to be more viable for many 
States. 

3. These changes should be made in time for the State agencies to be given 
a one-year transition period (Program Year 1992) prior to full 
implementation. 

4. The U.S. Department of Labor should provide technical assistance to help 
the States not currently using UI wage-record data. 

In eight States the JTPA office has not requested wage-record data. In several other States, these 
data were requested and because of one or more barriers, the agreement was not achieved. These 
States may need assistance in one or more areas, since data matching might be difficult without 
major changes in the JTPA or wage-record data systems. Barriers or obstacles that require 
attention include confidentiality requirements and lack of access to individual participant data 
in JTPA management information systems. 

a. States where changes are needed to overcome confidentiality barriers will need 
technical assistance. 

Data sharing agreements exist with ali 'evels of government, other States, universities and with 
private sector entities. However, the extent and conditions of access to wage-record data vary 
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among States as governed by a myriad of Slate and federal laws. While many States have dealt 
successfully with the confidentiality issue, five wage record agencies were unable to meet JTPA 
requests for wage-record data for confidentiality reasons. The development of better safeguards 
for confidentiality of the wage-record data should improve access to them. 

The VS. Department of Labor has recently released proposed regulations which govern the 
exchange of State wage-record data. These proposed regulations, published for comment on 
March 23, 1992 (57 FR 10064), address allowable reasons for sharing data, safeguards, cost 
reimbursement, oversight, and other confidentiality and disclosure matters. Although these 
regulations provide many of the elements necessary for a sound and complete data sharing 
agreement, the regulations do not intend to provide a specific data agreement model that States 
could use. 

States may need to proceed further to collectively build a common framework for data sharing 
that provides access equity to JTPA data users simultaneously with confidentiality safeguards 
to data providers. Model data sharing agreement language that is consistent with the finalized 
UI regulations could be developed to assist the States* 

b. States that have to make changes to data systems and enhance their computer 
capacity will need technical assistance- 

Those States that do not have access to individual participant data in their JTPA management 
information systems are likely to need technical assistance. Five of the responding States lack 
such access; in three of them, it would be difficult and time consuming to get the individual 
records from the SDAs. 

In four States with no data sharing agreements, wage record agency officials reported that it 
would be difficult for their offices to meet any additional data sharing requests and impossible 
if the volume of requests were to increase by 25%. Several States are operating computet* at 
their maximum capacity or close to it making even a small increment to the work load 
impossible to accomplish without additional resources. The average percentage of computer 
time devoted to processing data matching requests was 2.3%, with a range from as little as .05% 
to as much as 10.0% of agency computer capacity. 

c. States will need technic! assistance in addressing the out-of-state employment 
issue. 

While only an average of 3.5% of JTPA terminees residing in one State are placed in jobs outside 
that State's boundaries, some State JTPA officials indicated that this is a problem for some of 
their SDAs. No State, however, reported SDAs with out-of-State employment of 10% or more* 
Even though the impact of out-of-State employment for the overall JTPA system seems small, 
equitable treatment of affected SDAs in the distribution of incentive funds remains an issue. 
Interstate data sharing agreements are a direct approach to dealing with this out-of-State 
employment issue. An alternative approach would be for the U.5, DOL to develop a 
performance standards adjustment mechanism to account for out of-State employment. 

d. States will need technical assistance to address data accuracy issues. 

Inaccuracies can occur in data collected from both survey-based and administrative data 
sources. A sample of telephone survey responses are verified. The wage-record date on the 
whole are not systematically edited and usually are verified only when a claim is filed for 
unemployment benefits. 
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In general, the wage-record data received by State JTPA officials are perceived to be accurate. 
Several State JTPA officials reported that they encountered and corrected data errors such as 
duplicate wage records or unrealistic wages. Because JTPA offices that have not used 
wage-record data may have concerns with data accuracy, technical assistance that builds on the 
experiences of other States would be beneficial 

e. The li.S, Department of Labor should facilitate technology transfers and provide 
limited start-up funds to States to establish a UI data sharing system and to provide 
technical assistance to SDAs. 

In three States the JTPA office has not been able to forge a data staring agreement primarily 
because of cost constraints. The finance issue is particularly important to small States with 
limited administrative dollars. The experiences and technical knowledge of the States currently 
using wage-record data for JTPA purposes should be transferred to those States starting out to 
develop such a data sharing arrangement 

STATE/LOCAL PROGRAM MANAGEMENT AND EVALUATION 

5. The combined use of UI wage-record data and participant survey data 
should be considered lor purposes of program management/ gross 
outcomes evaluation, and net impact analysis. 

Participant surveys ami UI wage records each have certain advantages. For example, the current 
post-program survey provides more data about participants' employment experiences than the 
wage records for a single quarter. However, the survey is collected at only a single point in time, 
whereas the v age records could provide individual earnings histories for many quarters before 
and after p ogram participation. Survey data often suffer from nonresponse biases (to be 
examined in Chapter IV), whereas the earnings data from the wage records are difficult to 
compare across industries, employers and States for a variety of reasons discussed earlier. 

In conclusion, the combined use of UI wage-record data and participant survey data would 
provide a rich resource for JTPA program management and evaluation purposes. 



76 



Endnotes To Chapter II 



1. John Baj and Charles E. Trott A Feasibility Study of the Use of Uj Wage-Record Data as an 
Evaluation Tool for ITPA. Phase 1. National Commission for Employment Policy, Washington, 
D.C., Research Report Number 90-02, January 1991 . 

2. Jose Figueroa and Allan Misch, ITPA State Evaluation Reports, National Governors' 
Association, Washington, D.C, July 23, 1990. 

3. Thomas Plewes, "Labor Force Data in the Next Century," Monthly Labor Review, US. 
Department of Labor, Bureau of Labor Statistics, Volume 113, Number 4, Washington, D.C, 
April 1990. 

4. Hours worked: Washington. Weete worked: California, Iowa, Minnc^sota, Ohio, Oregon, 
Pennsly vania, and the Virgin Islands. 

5. Connecticut, Indiana, Iowa, Mississippi, Oregon, Puerto Rico, Virginia and the Virgin 
Islands. 

6. Nancy Bross, "Using UI Wage-RA:ord Data for JTPA Post-Program Performance Standards," 
U.S. Department of Labor, Washington, D.C, July 1991. 

7. Thomas Pie wes, 'Confidentiality," paper presented at a U.S. Bureau of the Census Research 
Conference, Washington, D.C, 1986. 

8. Ibid. 

9. John Baj and Charles E. Trott, A Feasibility Study . 

10. Based on information from 14 States. 

1 1 . John Baj and Charles E. Trott, A Feasibility Study . 

12. California, Indiana and Utah. 

13. John Baj and Charles E. Trott, A Feasibility Study . 

14. Colorado, Florida, North Carolina. 

15. Alaska. 

16. John Baj and Charles E. Trott, A Feasibility Study . 

17. Nancy Bross, "Using UI Wage-Record Data." 

18. Florida. 




-1 r~ 



77 



APPENDIX II.A 



State Responses To The Canvass 



79 



STATE RESPONSES TO THE CANVASS 





Both Rftpflftfft 


TTPA Onlv 

A mm Am VVuT 


UIQnlv 


AT ARAMA 


X 






AI JVSKA 1 


x 






ARIZONA * 


X 






ARKANSAS 


Y 
A 








x 






COLORADO *_2 




x 




rrWNFTTTCT IT 






x 

A 


DELAWARE ♦ 


X 






DISTRICT OF COLUMBIA 




x 




FLORIDA * 


x 






GEORGIA * 




x 




HAWAII 


x 






IDAHO * 




x 




ILLINOIS * 


x 






INDIANA * 


x 






IOWA * 






x 


KANSAS 


x 






KENTUCKY * 


x 






LOUISIANA 






X 

A 


MAINF 






X 

A 


MARYI AND * 




X 
A 




MASSACHUSETTS ^ 




V 
A 






X 

A 








A 






MTSSTQSTPPT 


X 

A 






MTQCOTTRT * 


X 
A 






MONTANA * 






Y 

A 


NEBRASKA M 


x 






NEVADA • 


X 






NFW HAMPSHIRE * 


x 






NEW JERSEY »,5 


X 






NEW MEXICO 


X 






NEW YORK 6 


X 






NORTH CAROLINA V 


X 






NORTH DAKOTA 






X 


OHIO * 


X 






OKLAHOMA 






X 


OREGON * 


X 
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TTPA Oniv 


TjIOnlv 


PENNSYLVANIA * 


A 






PUERTO RKX) 


V 
A 






RHODE ISLAND 






x 


SOUTH CAROLINA * 


A 






SOUTH DAKOTA * 


A 






TENNESSEE *,4 


V 
A 






lfcXAb * 


Y 
A 






f ?T A ur * 

UTAH * 


Y 
A 






VERMONT * 






x 


VIRGIN ISLANDb 






X 


VIRGINIA * 


Y 
A 








x 






WEST VIRGINIA 


X 






WISCONSIN 


X 






WYOMING 






X 


TOTAL NUMBER 


36 


6 


11 



• -Sharing Ul data with State JTPA office. 

1 -Sharing UI data on a project by project basis. 

2 -UI data used only to verify EDWAA eligibility. 

3 -Quarterly wage data collected by the Revenue Department 

4 -UI data used only to Identify potential JTPA participants. 

5 -UI data used to validate 13th-week postprogram telephone survey. 

6 -Quarterly wage date collected by the State Departtnent of Taxation and Finance. Wage data have been provided to some 
SDAs through agreements with the State Department of Labor, The data exchange has been for specific targeted populations. 
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Information Exchange Agreement between 
Texas Employment Commission and 
Texas Higher Education Coordinating Board 



The Texas Employment Commission, hereinafter referred to as TEC, and the 
Texas Higher Education Coordinating Board, hereinafter referred to as 
THECB, do hereby agree to the following: 

t Statement of Purpose 

As part of the "Lone Star Completer Follow-up Pilot Study," the 
Texas Higher Education Coordinating Board must survey those 
students who have attended the institutions under the purview of 
the THECB to determine the effectiveness of training received at 
those institutions. To conduct this survey, THECB must obtain the 
current location of former students. As part of the reporting system 
for the Unemployment Insurance system in the state of Texas, the 
Texas Employment Commission maintains a database of reported 
wages and employers, by Social Security Number, for a five-calendar 
period, not including the current or the most recently completed 
calendar quarter. The purpose of this agreement is to provide THECB 
with a listing of current employers of former students by matching a 
listing of Social Security Numbers of students against TEC's wage 
record file. 

H. Services 

At least once per quarter, THECB wiH submit to TEC a magnetic tape 
containing the Sodal Security Numbers of former students. TEC will 
conduct a computer match of this tape against its wage record 
database, providing THECB with a listing of the most current 
employer name and address for each S3N match. 

HI. Charges and Billing 

Following each computer match, as described in Section II above, 
THECB will be billed by TEC for the cost of providing this service. 
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Charges will be consistent with the computer utilization and staff 
charges in effect at the time the services are performed. Current 
charge rates are reflected in Attachment A; however, TEC reserves 
the right to revise the rates found in Attachment A at any time, with 
or without notice, based on TEC's actual costs. THECB agrees to pay 
the rates in effect at the time the service is performed. 

Payment will be made by THECB within 60 days of receipt of a billing 
by TEC for services provided under this contract. 

Confidentiality and Security of Data 

Employment data contained in TEC's database is confidential by law. 
This information is not to be published or made open to public 
inspection in detail form. Any use or redistribution of this data by 
THECB must be consistent with the purposes outlined in Section I, 
above, and must be expressly conditioned on an agreement with the 
recipient to maintain the confidentiality of the data so disclosed. 

THECB agrees to maintain sufficient safeguards under its procedures 
to prevent unauthorized access to confidential information provided 
to i* under this agreement and to prevent redisdosure of such 
confidential information, except as provided for in the foregoing 
paragraph. 

Waiver of Liability 

Wage record information on file with TEC is generally unverified 
until such time as an individual files a claim for unemployment 
benefits based on reported wages. For this reason, it should be deai!y 
understood that TEC's data disclosed pursuant to this agreement may 
contain inaccuracies due to erroneously reported information and/ or 
clerical error in transcribing this information. TEC makes no 
guarantee regarding the accuracy of this information. 

Legal Authorities 

The parties to this agreement are granted the authority to enter into 
this agreement by Article 4413 (32) V.T.C.S. TEC's contracting 
authority may be found at Tex. Rev. Civ. Stat. Ann., Art. 5221b-l et 
seq. THECB's contracting authority may be found at 
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VII. Mollification or Termination of Agreement 

This agreement may be modified at any time by the mutual, written 
consent of both parties. Either party may terminate this agreement at 
any time, upon written notice to the other party. 

VHL Execution 

For the faithful performance of the terms of this agreement, the 
parties, by their authorized representatives in their capacities as stated 
below, affix their signatures and agree to be bound by the provisions 
hereof. This agreement has been made and executed by me parties in 
duplicate. The effective date of this agreement will begin on the date 
of last signature by the executing parties, below. 



Approved by: 

Texas Employment Commission 

William Grossenbacher 
Administrator 

Signed in Austin, Texas 

This 2* day of T^^^y. 1991 



Higher Education Coordinating Board 




Dr. Kenneth Ashworth 
Commissioner 

Signed in Austin, Texas 

This Q. day of 1&*a^. 1991 
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TEXAS EMPLOYMENT COMMISSION 
JOB ACCOUNTING SYSTEM RATE WORKSHEET 



Typ« of Units 

CPU TIME 
JOBS RUN 

MEMORY USED 

DISK I/O 
DISK SPACE 

TAPE I/O 
TAPE MOUNTS 

TAPE STORAGE 

STANDARD PRINT 

SPECIAL PRINT 

PROGRAMMER/ 
ANALYST TIME 

NON -PROGRAMMING 
STAFF TIME 



Description of Unlfr 

Number of seconds of computer processing time. 

Number of jobs, TSO logons, etc. run during the 
monthly accounting period. 

Amount of main storage occupancy In hundreds of 
CPU page seconds used by Jobs. 

Number of disk read and write operations X 1000. 

Amount of disk space required to perform job functions 
and store permanent flies, measured in megabyte hours. 

Number of tape read and write operations X 1000. 

Number of times tapes manually placed on tape drive 
equipment 

Number of days tape(s) are held in tape library 
or security vault. 

Number of pages printed on Xerox 9700 printer. 

Number of pages of manufacturers preprinted forms. 

Per hour rate for any necessary programmer/analyst 
time. 

Per ho-r ra**» for staff time spent in connection with 
processing of a request, other than programming time. 



fiAlflft 

00.231218 

02.374857 

00.237207 
00.219742 

00.006584 
00.268429 

01.065082 

00.065967 
00.011637 
00.128476 

27.59 

17.50 



The above rates are periodically revised to take into account any significant changes in certs of 
staff, equipment, system software, etc. Once revised, these rates are automatically applied to all 
computer utilization jobs. 
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DES Number 



ES-90-91-2645 



COMMONWEALTH OF KENTUCKY 
CABINET FOR HUMAN RESOURCES 
DEPARTMENT FOR EMPLOYMENT SERVICES 
DIVISION OF UNEMPLOYMENT INSURANCE 

AND 

DEPARTMENT FOR SOCIAL SERVICES 



THIS AGREEMENT, made and entered into as of the first day of 
July, 1990, by and between the Department for Employment Services, 
hereinafter referred to as DES, and the Department for Social 
Services, hereinafter referred to as DSS. 



WHEREAS the Department for Social Services (DSS) has the 
responsibility for determining Title IV-E eligibility for foster, 
children pursuant to 45 CFR, Part 1357. 

WHEREAS in discharging its responsibility the DSS has need of 
information upon the residence and employment of responsible 
parents; and 

WHEREAS the Cabinet for Human Resources, Department for Employment 
Services, (hereinafter DES), Division of Unemployment Insurance 
routinely maintains, in the administration of the unemployment 
compensation programs, information upon the addressed employment 
status of those who either (1) receive unemployment benefits «w (2) 
work in a job covered by the Unemployment Insurance Act; and 

WHEREAS pursuant to KRS 341.190, information within the files of 
the Division of Unemployment Insurance, though conf iden:ial, may be 
shared with other governmental agencies when used in the course and 
scope of the agency's responsibllty and upon adequate assurances of 
confidentiality; and 

WHEREAS the sharing of this confidential information with DSS would 
assist that agency in carrying out its responsibilities, and would 
further foster efficient economical government; 

NOW THEREFORE it is hereby agreed by and between the DES and DSS as 
follows: 

1. The Division of Unemployment Insurance ..hall provide to 
the DSS information upon the names and addresses of 

i ft if- ****** ;-' of Unemployment Insurance benefits; 



WITNESSETH THAT: 
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It Is further agreed that the Division of Unemployment 
Insurance shall provide information within its files on 
individuals in covered employment (consisting of at least 
the names and addresses of employers) which it routinely 
collects during its administration of the Unemployment 
Insurance Program; 

The Division of Unemployment Insurance shall provide the 
foregoing types of information by providing on-site 
access through user terminals to programs 4B, 4C, 4E, 40, 
43, and 48. 

A list of users in DSS will be provided to DES and kept 
current by DSS by immediately notifying DES of any such 
employees who terminate their employment with the Agency 
or whose job duties no longer require access to DES files. 
Further, all employees who have access to unemployment 
insurance databases are required to read, sign and 
forward a copy to DES 2 security statements, "Access to 
On-line Computer Screens" and "Security Agreement for All 
Employees". (See Attachments A and B). 

The Agency agrees to provide a written copy of their ^ 
internal security safeguards to insure that information 
obtained from DES will be protected against unauthorized 
access or disclosure and as a minimum agrees to comply 
with 20 CFR 603.7 which reads as follows: 

"§ 603.7 Protection of confidentiality. 

(a) State unemployment compensation 
agencies shall require requesting agen- 
cies receiving information under this 
Part to comply with the following mea- 
sures to protect the confidentiality of 
the information against unauthorized ac- 
cess or disclosure: 

(1) The information shall be used only 
to the extent necessary to assist in the 
valid administrative needs of the pro- 
gram receiving such information and 
shall be disclosed only for these pur- 
poses as defined in this agreement; 

(2) The requesting agency shall not 
use the information for any purposes not 
specifically authorized under any agree- 
ment that meets the requirements of sec- 
tion 603.6; 

(3) The information shall be stored in 
a place physically secure from access by 
unauthorized persons; 



(A) Information in electronic format, 
such as magnetic tapes or discs, shall 
be stored and processed in such a way 
that unauthorized persons cannot re- 
trieve the information by means of com- 
puter, remote terminal or other means; 
(5) Precautions shall be taken to en- 
sure that only authorized personnel are 
given access to on-line files; 
(6>(i) The requesting agency shall in- 
struct all personnel with access to the 
information regarding the confidential 
nature of the information, the require- 
ments of this Part, and the sanctions 
specified in State unemployment compen- 
sation lavs against unauthorized dis- 
closure of information covered by this 
Fart, and any other relevant State stat- 
utes, and 

(ii) The head of each State agency shall 
sign an acknowledgment on behalf of the 
entire agency attesting to the agency's 
policies and procedures regarding con- 
fidentiality. 

(b) Any requesting agency is authorized 
to redisclose the information only as 
follows: 

(1) Any wage or claim information may 
be given to the individual who is the 
subject of the information; 

(2) Information about an Individual 
may be given to an attorney or other 
duly authorized agent representing the 
individual if *he individual has given 
written consent and the information Is 
needed in connection with a claim for 
benefits against the requesting agency; 
and 

(3) Any wage or claim Information may 
be given to another requesting agency as 
defined in whis Part or to any crimi- 
nal or civil prosecuting authorities act- 
inn; for or on behalf of the requesting 
agency if provision for such redisclo- 
sure is contained in the agreement be- 
tween the requesting agency and the 
State unemployment compensation agency. 

(c) The requesting agency shall permit 
the State unemployment compensation 
agency to make on site inspections to 
ensure that the requirements of State 
unemployment compensation laws and 
Federal statutes and regulations are 
being met (sec. 1137(a)(5)(B)." 
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6. Pursuant to 20 CP! 603.7(b)(3) stated above, the State 
unemployment compensation agency does not grant permission 
of redisclosure of any wage or claim information to 
another requesting agency that is not a party to this 
agreement, or to any criminal or civil prosecuting 
authorities acting for or on behalf of the Agency. 

7. The period within the current fiscal year in which this 
agreement is in effect is from July 1, 1990, to June 30, 
1991, and may be renewed by agreement of the parties. 

8. Either party may terminate this agreement upon thirty 
(30) days written notice to the other or immediately for 
cause . 

9. This agreement may be modified or amended by mutual 
agreement of the Parties hereto. 



DEPARTMENT FOR EMPLOYMENT 
SERVICES 



By 



Commissioner 



DEPARTMENT FOR 
SERVICES 



SOCIAL 



By 



tfja^issVoner 



EXAMINED AS TO FORM 
ANIL SUBSTANCE: 





Attorney 
Office of the 



APPROVED 




Human Resources 



hi 
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ATTACHMENT A 



Access to On-line Computer Scrggag 

All computerised flit, »rt restricted to authorized users. Access to eh... 
file, can be clued only through .utering both operator ID «od peiswrd 
Each person's operator ID and password are confidential and cannoT be 
it™}*** t0 " yoni Eluding managers and supervisors. Any employe. 

his/her operator password will bs subj.ct to immsdlat. dl.clpUnl" 

rteT. X r«?n.r^ !22? «?' 'T'" iCr " n B£*££ to leaving 

ens terminal to avoid misuse by another. Any employee who l..v.. thl 

P * l0r t0 "•^•-«« M *U *>• «ubject y «rJSc^tt di^iJS 

sigiisture on this sheet certifies that 1 have read and understand the 
above policy ststement and will adhere to the terms of the policy. 



Employee Signature Date of Certification 



My signature on this sheet certifies that I have discussed the above policy 
with this employee, and explained the Importance of abiding by the policy. 



Supervisor Signature Date of Certification 
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Attachment B 



Department for Employment Services 
Frankfort, Kentucky 

Security Agreement for all Employees 



LOCATION: 



j , hereby acknowledge that I fully 

understand the rules, regulations, and statutory law regarding confidentiality 
of claimant and employer records and other activities necessary in the 
administration and implementation of the unemployment insurance program. KES 
341.190(3) reads as follows: All letters, reports, communications, ana other 
matters, written or oral, from employer or worker, to the secretary for human 
resources or any of his agents, representatives, or employees, or to the 
commission, or to any board or official functioning under this chapter, which 
have been written, sent or made in connection with the requirements and 
administration of this chapter, shall be absolutely privileged and 
be the subject matter or basis for any suit for slander or libel in any court, 
but no employer or employee, or his representative, testifying before the 
commission, the secretary for human resources or any duly authorized 
representative thereof, shall be exempt from punishment for Perjury. 
Information thus obtained shall not be published or be open for public 
inspection except to public employees in the performance of their duties, in 
any manner revealing or indicating the employing unit's identify, but any 
claimant at a hearing before the commission, the secretary for human resources 
or his duly authorized representative, shall be supplied with information from 
such records to the extent necessary for the proper presentation of his claim. 

I also understand the penalties which can be imposed for violations of any 
rule, regulation, or law and fully understand the consequences if I violate 
any rule, regulation or law. KSS 341.990(1) reads as follows: Any employee 
of any state department who violates any of the provisions of KRS 341.110 to 
341.230 shall be fined iot less than twenty dollars ($20.00) nor more than two 
hundred dollars ($200) or imprisoned for not more than ninety (90) days, or 
both. 



- Signature of Employee Date of Certification 

My signature on this agreement certifies that I have discussed the meaning of 
this agreement with this employee, explained the importance of abiding by the 
policy, and the penalties for failing to do so. 



ERIC 



Supervisor's Signature Date of Certification 
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APPENDIX H.C 



MSAs Sharing State Boundaries 
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MSAs SHARING STATE BOUNDARIES 



METROPOLITAN STATISTICAL AREA 


1988 POPULATION 


Allentown-Bethlehem, PA-NJ 


677,000 


Augusta, GA-SC 


396,000 


Boston-Lawrence-Salem. MA-NH 


4,110,000 


Chartotte-Gastonia-Rock Hill f NC-SC 


IM2JB0Q 


Chattanooga, TN-GA 


438,000 


Chicago-Gary-Lake County (ID, IL-IN-WI 


8481,000 


Gncinnati-Hamilton, OH-KY-IN 


1728,000 


Davenport-Rock Island-Moline r IA-IL 


364,000 


Evansville,IN-KY 


28 L000 


Huntington-Ashland, WV-KY-OH 


322,000 


Kansas City, MO-KS 


1,575,000 


Louisville, KY-IN 


967,000 


Memphis, TN~ AR-MS 


979,000 


Minneapolis-St Paul, MN-WI 


2388,000 


New London-Norwich, CT-RI 


259,000 


New York-Northern New Jersey-Long Island, NY-NI-CT 


18,120,000 


Philadelphia-Wilmington-Trenton, PA-NJ-DE-MD 


5,963,000 


Portland-Vancouver, OR-WA 


1,414,000 


Providence-Pawtucket-Fall River, RI-MA 


1,125,000 


St. Louis, MO-IL 


2,467,000 


Washington, DC-MD-VA 


3,734,000 
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CHAPTER III 

The Confidentiality Provisions Of 
State Unemployment Compensation 
Laws 



Introduction 



Three years ago theN itior al Commission for Employment Policy commenced a feasibility study 
on using Unemployment Insurance (UI) wage records, rather than telephone surveys, as a way 
to gather information on the long-term employment and earnings of participants in programs 
funded under the Job Training Partnership Act (JTPA',. 1 At the outset, 11 States participated in 
the field aspects of this project. This number grew to 15 in a second phase, and to 20 during the 
pa st year. 

One of the issues that emerged in this project was how to assure the confidentiality of 
employment and earnings information acquired from a State Employment Security Agency 
(SESA). Information on a former JTPA participant's employment and earnings can only be 
extracted from a State's UI wage records if an accurate social security number appears in both 
the JTPA and UI files. The Commission's Phase I report included the following statement 

Abuse of confidentiality is not inevitable. Awareness of the privacy standards 
and of the sanctions that will be imposed for violating them will control most 
of the vulnerability of the data. Common sense caution in the handling of 
records will eliminate most of the risk that remains. It is reasonable to assume 
that the heightened awareness of the issue will lead to tighter security. 2 

A contractor report prepared for a July 1991 technical workgroup meeting convened by the U.S. 
Department of Labor's Employment and Training Administration, Division of Performance 
Management and Evaluation, stated th-»t 

State employment security agencies have a great deal of latitude in how they 
administer their unemployment insurance programs. This latitude give* 
them flexibility in coordinating their activities with those of other programs, 
but it also means that data access agreements and logistics must be worked 
out at the State level. Data access is governed by both federal and State laws 
and regulations regarding confidentiality. 3 

This chapter documents the results of an investigation of the States' confidentiality laws and 
administrative practices. Conceptual, legal and management issues are given balanced 
attention. 
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The relevance of State confidentiality laws and administrative practices for JTPA-spedfic 
applications is best understood by placing this use in an historical context The Deficit Reduction 
Act of 1984 stipulated that 

employers in leach] State are required, effective September 30, 1988, to make 
quarterly wage reports to a State agency (which may be the agency 
administering the State's unemployment compensation law)... 

This amendment of the Social Security Act was explicitly intended to facilitate the verification 
of income and eligibility for selected federal benefit programs. 

At that time, three-fourths of the States were already collecting this information through St&te 
Employment Security Agency (SESA) auspices, in support of their State-authorized and 
administered unemployment compensation programs. At present, only Massachusetts and 
New York (using a Tax Department Wage Reporting System) do not collect the required 
information through the SESA. 6 Michigan's Employment Security Commission requires 
covered employers to submit quarterly wage reports only in compliance with State and federal 
laws requiring an Income and Eligibility Verification System. 

The nearly universal availability of wage record data has triggered a substantial increase in the 
number of requests that a SESA gets from external parties for access to these data. This torrent 
of requests, in turn, has resulted in an urgent need for each of the SESAs and the Unemployment 
Insurance Service in the Employment and Training Administration of the US. Department of 
Labor to determine the legal and administrative consequences of responses to these requests. 

The Unemployment Insurance Service has taken the lead within the VS. Department of Labor 
to develop a policy that protects the Service's compelling interest in the integrity of the original 
(and continuing) administrative purpose for collecting these data-the management of th| 
Nation's unemployment compensation system in cooperation with the individual States. 
Proposed regulations addressing this topic were published for comment on March 23, 1992 (57 
FR 10064). 

The U.S. Bureau of Labor Statistics has faced and resolved related issues for 100 years. 10 Today, 
the Bureau operates under the guidance of Commissioner's Order No. 2-80, which prescribes 
the rules for handling confidential Bureau records." Recently, the complexity of complying 
with this policy has been documented in a retrospective review of an Employer Reporting Unit 
Match Study. This research involved linking employer records maintained by the Bureau of 
Labor Statistics, the Social Security Administration and the Internal Revenue Service. One 
participant in this project recalls that "everything ... had to be invented. There are no textbook 
examples of interagency agreements on confidentiality. The solutions which the project team 
developed were carefully crafted to stay within the very restrictive IRS law and were 
implemented with an eye toward the reality of the environment" 

Discussing a paper titled "Disclosure Avoidance Practices at the Census Bureau," George T. 
Duncan observes that 

... essentially the task is a decision-theoretic one that incorporates the 
motivations of the stakeholders. Since these stakeholders are 
various-including individual respondents, other government agencies, 
academic researchers, market researchers, commercial planners, the media, 
and lobbying groups— the (disclosure limitation] measures developed should 
be multivariate in nature. 14 
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An important step in the crafting of useful textbook examples of interagency agreements on 
confidentiality, and in understanding stakeholder motivation, is to assure that all parties use a 
common language. The next section addresses this issue. 

Confidentiality Issues 15 

DIFFERENT PERSPECTIVES ON CONFIDENTIAL INFORMATION 

Reynolds offers three rationales for investigating the confidentiality issue: 16 

• public concern about the ability to maintain a "sphere of privacy* and avoid 
surveillance; 

• the anticipated or revealed distribution of costs (hams) and benefits (rewards) 
resulting from disclosure; and 

• respect for appropriate standards of interpersonal conduct. 

Reynolds also distinguishes three types of informed consent, which sharpen the discussion with 
respect to JTPA-specific applications: 

• active informed consent, which requires the well known conditions of rational 
judgement, full information about the "conditions associated with the decision", 
absence of coercion and full awareness of potential negative [emphasis added] 
consequences that could result from the decision; 18 

• passive informed consent, which is assumed in the absence of any contradictory 
action; and 

• proxy active informed consent, which allows an intermediate authority (eg. a 
government official) to make an explicit, or active, determination that passive 
informed consent is applicable. 

The relevance of these distinctions for the use of UI wage records by the JTPA system is 
apparent-the proxy active consent approach is the most frequent, although not universal, 
practice that has been adopted to date by the SESAs. 

'PRIVATE INFORMATION' AND HARM 

The SESAs and the Unemployment Insurance Service express a shared concern about the harm 
that might arise from public disclosure of confidential administrative information. 

Substantial state and private resources are expended and utilized in 
compiling, reporting, processing, and storing information with the [Utah] 
Department {of Employment Security]. The primary purpose for this 
extensive effort is to provide an unemployment insurance system by which 
unemployed individuals are paid benefits and employers are taxed to finance 
the program. Any policy governing the disclosure of wage and claim 
information must make allowance for the very important consideration of the 
potential chilling effect [emphasis added] disclosure will have on both 
employers, and claimants. 
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Reynolds identifies four categories of potential harm that might result from public disclosure of 
Information; 

(1) penetration of the privacy sphere; 

(2) disclosure that is personally embarrassing; 

(3) disclosure that results in a decrement in the individual's situation; and 

14) indirect loss of status that results from a group affiliation or identification. 

With respect to the use of UI wage records within JTPA, the first and fourth of these categories 
appear to be more relevant than the second and third; and the privacy sphere issue is relevant 
only if a person s identity may readily be ascertained by an investigator. 

One intended side-effect of using UI wage records for JTPA purposes is to affect the future status 
(i.e., public image) of past, present and future JTPA participants. However, the possibility of 
indirect harm must be weighed against two other considerations: 

• the possibility that image enhancement will result from the availability of better 
management information; and 

• the previously cited moral principle of "adherence to appropriate standards for 
interpersonal conduct." 

Reciprocity is an important element in the social contract If information about employment 
and earnings can be used without revealing the identity of any person or business to improve 
the management of a public program, men there is no potential chilling effect on the States' 
unemployment compensation systems. 

RATIONALES FOR ORGANIZATIONAL CONFIDENTIALITY 

Reynolds offers two rationales for organizational confidentiality: 

• "the economic conception of organizations as autonomous agents in competitive 
markets/' which focuses on the need for confidentiality to permit equitable 
competition; and 

• the fact that "modem societies are now seen as societies of organizations, or juristic 
persons, where most relationships between the individual and the state are mediated 
by organizations ..." 20 

There is nothing in the JTPA use of UI wage records that appears to threaten the competitiveness 
of the organizations that are required to submit the information to a SESA. Indeed, it appears 
that "past research access to many privileged, confidential and anonymous data sets in the 
United S'ates has not produced a single case of harm involving the breach of the anonymity, 
confide- ice, or privacy of any respondent." 21 

There does not appear to be any conceptual, ethl&al or mora' barrier to the use of wage records 
within JTPA, so long as individual and organizational identities are not revealed to the public. 
The next section provides a bridge from abstract ethical and legal principles to day-to-day 
practice in SESAs. 
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State Laws 



There is nearly universal core content in the provisions of State laws that regulate SESA records 
and reports. Appendix IDLA contains a summary of these provisions. 

Each employing unit is required to keep accurate records containing the information that is 
necessary to administer each State's law. The subsequent handling of this information is 
controlled by the uniform stipulation that it shall be held confidential, except to the extent 
necessary for the proper presentation of the contest of a claim, and shall not be published or 
be open to public inspection, other man to public employees in the performance of their 
duties, in any manner that reveals an individual's or employing unlfs identity. The words 
"published", "open to public inspection" and "public employees in the performance of their 
duties" are obviously subject to interpretation. The consequences of this are discussed in the 
following pages. 

Stated briefly, a thorough review of State laws and administrative documents, personal 
experience in attempting to acquire wage-record data from individual State agencies, and 
statements of interested colleagues throughout the country, indicate that 

• no State law explicitly bars the release of wage-record data by a SESA to the 
administrators of a Job Training Partnership Act program; 

• a few of the State laws explicitly authorize the release of wage-record data by a SESA 
to the administrators of a Job Training Partnership Act program (e.g., Arkansas, Iowa, 
Minnesota and West Virginia); 

• most of the State laws grant the SESA's executive officer discretionary authority (i.e., 
proxy active consent) to release wage-record data for specific program purposes that 
meet the SESA's own confidentiality standards and that carry the SESA's sanctions- 
and 

• a few of the State laws are silent with respect to legislative intent regarding this release 
of administrative records. 

The following are State-specific complements to the common core wording o> confidentiality 
stipulations. They exemplify what a thorough State-by-St ate examination of the statutory 
provisions reveals: 1 

• assignment of agent status to designated third parties (AZ, CO and KY); 

• authorization of research uses of the data (AR, CA, MI, NJ and OR); 

• JTPA-specific authorization (AR, I A, MN and VW); 

• administrative discretion granted (AL, DL, GA, HA and PA); and 

• specified exceptions only (MN and OH). 

Other State laws include different combinations of these basic features. 
Theassignmentofagentstatosiliustratestheconcer^ofpro 

practice. Third-parties are given the authority to act as responsible custodians of an agency's 
administrative records. Here, intermediate entities are not viewed as the public from whom 



er|c BEST COPY AVAIUil m * H 



information is to be withheld. Instead, they are given legal status to stand between the SESA 
and the public, i.e., they are an extension of the SESA for this purpose. 

State-specific authorizations of research uses of administrative records can be placed on a 
continuum: from legislative approval of internal research use, through unspecified agents, to 
Arkansas' designation of a single university campus! 

New Jersey's administrative policy illustrates the intent of these provisions. 

The Division of Unemployment and Disability Insurance will permit the 
release of certain data to public and private organizations such as colleges, 
universities or foundations to perform research which can be expected to 
benefit the State's residents. The administrator of the agency engaged in the 
research must certify in writing that the confidentiality of the disclosed 
information will be maintained. Usually, the Department of Labor will draft 
a specific agreement whereby the research facility agrees to comply with the 
statutory and regulatory provisions relating to confidentiality. 

The remaining three types of core language extension^ 1) JTPA-specific authorizations; (2) 
granting of administrative discretion; and (3) restriction to specified exceptions only-represent 
a continuum of flexibility from "always", through "maybe", to "never", if JTPA is not one of the 
specified exceptions. 

In the next section, the State laws are complemented by an examination of administrative 
documents that were submitted to the author by 26 of the SESA administrators. These 
documents reflect the administrators' efforts to translate law into everyday practice. 

Administrative Documents 



Arkansas' Attorney General recently offered the following opinion to the Administrator of 
Arkansas' Employment Security Division: "... The existence of a compelling public interest in 
disclosure must be determined on a case by case basis. As a general matter, however, legislative 
intent is that public disclosure is favored and nondisclosure is the exception. ... 

This contrasts with a much earlier Nevada Attorney General's opinion that "where legislature 
has provided exceptions to operation of statute, there is presumption that no other were 
intended",* 24 and with the Utah Department of Employment Security Chief Legal Counsel's 
statement cited in endnote 19 of this chapter. 

The Purpose paragraph of the Ohio Bureau of Employment Services' Administrative Directive 
on confidentiality states that "the policy stated here is a significant change from earlier policy 
and is the result of amendments to the Ohio Unemployment Compensation Law passed by the 
General Assembly in 1989." 25 This document offers the most comprehensive statutory 
references and review of policy considerations among those that were reviewed. In a 
Background section the Directive states that "previous bureau policy on confidentiality strictly 
prohibited release of employer and individual information except for purposes specifically 
required by federal or Ohio law. Revisions to the Ohio Revised Code passed in 1989 provide 
authority to the administrator to release or exchange confidential information for purposes of 
employment and training." 26 

In a section titled Principles of the Confidentiality Policy, an Attachment to the Administrative 
Directive states that "the bureau recognizes that it has different and sometimes conflicting 
responsibilities with respect to confidential information. The bureau has a responsibility to 
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employers and individuals who provide the information ... Ths bureau also has a responsibility 
to use its resources, including information, to reduce unemployment and to provide and 
improve employment and training services. ... As a means of balancing these responsibilities, 
the bureau adopts the following principles regarding confidentiality of information: 1. It is the 
policy of the Bureau of Employment Services mat confidential information shall {emphasis 
added] be released only for purposes of providing employment and training services or as 
otherwise required or permitted by federal or state statute. ... Releaseof confidential information 
otherwise permitted and not required by federal or state statute shall be at the discretion of the 
administrator [emphasis added] in accordance with section VI below." 27 This document further 
states that "li]t is the policy of the Bureau of Employment Services that any request for release 
of confidential information that is not specifically required or permitted by federal or state 
statute, or that is not clearly for purposes of employment and training services, shall be denied 
[emphasis added]." 28 

The Utah Department of Employment Security's administrative document 29 carefully reviews 
legal and policy considerations and offers recommendations based on specified requirements 
for disclosure of information and requirements for safeguarding disclosed information. 30 

These administrative interpretations exemplify the commitment of the individual SESAs to 
achieving a workable balance between individual and organizational rights and sodetal 
interests. 

Conclusions 



Both the State laws and administrative documents that have been examined demonstrate that 
the Nation's unemployment compensation system is made up of basically autonomous SESAs 
and a federal Unemployment Insurance Service. Interstate differences, similar to those that have 
been described here can be expected to persist, although the specific nature of these differences 
will change. 

The purpose served by (1 ) this chapter, (2) the National Commission for Employment Policy's 
laiger project on the feasibility of using I 'I wage records within the JTPA system, and (3) the 
Unemployment Insurance Service's efforts to develop a practical policy that balances often 
competing interests, is to inform the SESAs and related interested parties about the actions 
various parties have taken to deal with a common problem. 

No one should expect that an improved flow of information among the States will lead to a quick 
or total convergence of State laws or administrative practices. However, the differences that 
remain will more accurately reflect genuine disagreements about priorities among competing 
claims regarding confidentiality and disclosure. 

With respect to potential uses of Unemployment Insurance wage records within the JTPA 
system, this chapter offers ample reference to authoritative sources of information abo t how 
these records can be managed in full compliance with State laws and administrative regulations. 
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State-specific Language 



Statutory Provisions 

This Appendix presents State-specific language that refers to privacy and confidentiality 
requirements of the SESAs' enabling legislation. All of the States are not cited here because there 
is a substantial amount of uniform, or nearly identical, language in the individual statutes. 

State-specific language is introduced in a building-block manner, which is intended to highlight 
the variety of approaches that have been taken toward a common goal-to balance Individual 
and organizational rights to privacy or confidentiality and other agency-specific and societal 
interests. The States are presented in alphabetical order. 

Alabama's law reveals the first language that differs from the standard core content described 
in the body of the chapter. It provides that 

the director may, at hisdiscretion, release information regarding employment, 
wages, wage rates and unemployment to institutions of higher education of 
this state, or a federal governmental corporation upon payment of reasonable 
cost therefor, for the purpose of making economic analyses; provided, that 
such institution or corporation agrees that information so obtained will not be 
published or released by it to any person or persons in such manner as to 
permit the identification of any specific individual or employing unit. 1 

This is a statutory example of the concept of proxy active consent, in which public officials act 
as responsible custodians with respect to individual rights. It also indicates one legislative 
approach to interpreting the core concept of "release to the public". Here, intermediate analytical 
entities are not viewed as the public; they are given agent status by the SESA. 

Arizona'slawaddstothecorcdisclosurelan 

designated as such in writing for the purpose of accomplishing certain of the ctepartment's 
functions ... . No further legislative definition of these certain allowable functions is made. 
Arizona s law further provides that unit record information can be released "to agencies of the 
federal government, this state or any political subdivision of this state for official purposes. All 
information received by a governmental agency pursuant to this paragraph shall be maintained 
as confidential.' These provisions serve the same purpose as Alabama's law-they permit the 
release of administrative records to intermediate agents who are then required to respect the 
binding public release standards. 

Arkansas' law provides that information obtained from employing units "..shall be held 
confidential and shall be protected by government privilege.'* The concept of proxy active 
consent seems to mean that government privilege covers both the privilege to withhold and the 
pnv lege to release. This law further stipulates that "notwithstanding any other provisions of 
this la w, information obtained in the administration of this law shall be disclosed to the extent 
permitted in accordance with ... the JTPA, Act of 1982, Public Law 97-300 as amended, and 
implementing regulations promulgated thereunder by the United States Department of Labor 
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CalUomla'slawprovides that "...informationmay be tabulated and published in statistical fonn 
for the use and Information of state departments and the public, except that the name of the 
employing unit or of any worker shall never be divulged in the course of the tabulation or 
publication.-* This illustrates a legally defensible interpretation of the core concept of 
confidentiality that permits many uses of the administrative records. 

Colorado's law extends access to its records "...to an agent of the division designated a* such in 
writing for the purpose of accomplishing certain of the division's functions , whichoffers the 
agency discretionary authority to determine the scope of release that will be permitted. 

Delaware's law provides that "the Department shall disclose information, the reteaseof which 
is otherwise prohibited, to officials and employees of governmental agencies in the peifermance 
of their official duties, as It may by regulation permit [emphasis added!, provided: a. The 
regulation specifies the type of information to be released and the uses to which the iruormation 
nwDe put, coitsistent with theadn^^ 

legitimate governmental interests; c. In addition to the requirements of this paragraph, an 
other requirements with respect to the confidentiality of information obtained in the 
administration of this section and the sanctions imposed for improper disclosure shall apply 
to the use of such information by officials and employees of agencies to which information 
is released pursuant to this paragraph {emphasis added]. 

Georgia's law is explicit: "This article is intended to reconcile the free access to public records 
granted by Article 4 of Chapter 18 of Title 50, relating to the inspection of publicrecords,and 
the discovery rights of judicial and administrative systems with the historical confidentiality of 
certain records of the department and the individual's right of privacy. ...It is the intent of this 
article to define a right of privacy and confidentiality as regards individual and employing unit 
records and other records-land] it is the intent of this article to define also certain exceptions to 
the right of privacy and confidentiality." 9 The law further states that "The Commissioner shall 
have the authority to adopt, amend, or rescind rules and regulations interpreting and 
implementing the provisions of this article. In particular, these rules shall specify the procure 
to be followed to obtain information or records to which the public has access under this 
chapter." 10 Illustrative third-party uses of data are identified in the law, but these are not 
restrictive in the sense that other uses are prohibited. 

Hawaii's law provides that "subject to such restrictions as the director may by regulation 
prescribe, the information and determinations may be made available to : ...(4) Any other federal, 
state or municipal agency if the director deems that the disclosure to the agency serves the public 
interest." 11 however, the statute later stipulates that "no disclosure of information obtained at 
any time from workers, employers, or other persons or groups in the course of administering 
the state employment security program under .... shall be made directly or indirectly except as 
authorized by [subsequent sections of the law]. 12 A separate section covers disclosure ©f 
quarterly wage detail infonnation. This section provides in part: "(a) The department shall 
disclose quarterly wage detail information to authorized requesting agenrvs which have 
entered into an agreement in accordance with subsection (b) for purposes deemed by the 
department to be ...reasonably necessary for the proper administration of the requesting 
agency's program, (b) The agreement between the department and the requesting agency shall 
include, but not be limited to: ... lemphasis added and six provisions omitted here], (c) 
Requesting agencies shall comply with the following measures to protect confidentiality of the 
infonnation against unauthorized access or disclosure ... Isix provisions omitted here], (d) The 
requesting agency shall permit the department and the U.S. Department of Labor (and other 
authorized federal officials) to make onsite inspections to ensure that the requirements of this 
section are being met." 13 This section appears to offer language that could be used to amend any 
State statute in cases where Hawaii's intent is sharr d and a legislative approach is desired. 
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Iowa's law is explicit with regard to the relevance of a person's consent for the release of 
confidential information: "An open record is routinely disclosed without the consent of the 
subject. To the extent allowed by law, disclosure of a confidential record may occur without the 
consent of the subject. Following are instances where disclosure, if lawful will generally occur 
without consent of the subject: a. For a routine use ...{which is defined in a subsequent section 
to mean] the disclosure of a record without the consent of L.e subject, for a purpose which is 
compatible with the purpose for which the record was collected... To the extent allowed by law, 
the following uses are considered routine uses of all agency records: ... d. Direct disclosure of 
information with an attempt to provide notification to the subject and for a purpose consistent 
with Iowa Code chapter 96 to any of the following: ...(8) A political subdivision, governmental 
entity, or nonprofit organization having an interest in the administration of job training 
programs established pursuant to the federal Job Training Partnership Act" 1 * This appears to 
serve as a model for legislatively codifying approved uses of private information, where 
compatibility with the purpose for which the record was collected is the approval criterion that 
is invoked. r 

Kansas' law provides that "information shall be disclosed as required by any other statute of 
the federal government or the state of Kansas if the request for information is in writing and the 
statutory authorization for the release of the requested information is cited in the written 
request."" 

Kentucky's statute includes the phrase "information may be made available to public employees 
m the performance of their duties, but the agency receiving the information shall assure the 
confidentiality, as provided for in this section, of all information so released." 16 This is another 
approach to the concept of intermediate agent, upon whom the agency's own confidentiality 
standards are placed. ' 

Michigan's law provides that "subject to restrictions as the commission rrr.y prescribe, by rule 
or otherwise, the commission may also make such information available to colleges, universities, 
and public agencies of this state for use in connection with research projects of a public service 
nature. A person associated with such institutions or agencies shall not disclose the information 
m any manner which would reveal the identity of any individual or employing unit from or 
concerning whom the information was obtained by the commission." 1 ^ This repeats, in an 
explicit recipient context, the legislative establishment of an approved intermediary agent. 

Minnesota's law represents an explicit "exceptions only" approach: "Except as hereinafter 
otherwise provided, data gathered ...are private data on individuals or nonpublic data not on 
mdividuals and may not be disclosed except pursuant to this subdivision or a court order. 
These data may be disseminated to and used by the following agencies without the consent of 
the subject of the data: (a) state and federal agencies specifically authorized access to the data 
by state or federal law; ...(0 the department of labor and industry on an interchangeable basis 
with the department of jobs and training subject to the following limitations and 
notwithstanding any law to the contrary: (1) the department of jobs and training shall have 
access to private data on individuals and nonpublic data not on individuals for uses consistent 
with the administration of its duties ... 

Nevada's unemployment compensation statute contains, in an Annotations section, Attorney 
General s Opinions, one of which states that "where legislature has provided exceptions to 
operation of statute, there is presumption that no other were intended." 19 This strict adherence 
to an unobserved concept of legislative intent stands in sharp contrast to other State laws that 
give agency administrators wide latitude to exercise proxy active consent. 
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Oklahoma's law cont »ns a stipulation similar to Nevada's~"it is further provided that the 
provisions of this section shall * e strictly interpreted and shall not be construed as permitting 
disclosure of any other information convened in the records and files of the Commission. 

Oregon's law provides that "...information secured from employing units pursuant to this 
chapter may be released to agencies of mis state, ana ^litical subdivisions acting alone or in 
concert in city, county, metropolitan, regional or state planning to the extent necessary to 
properly carry out govemme 1 planning functions performed under applicable law. 
Information provided such agencies shall be confidential and shall not be released by such 
agencies in any manner that would be identifiable as to individuals, claimants, employees or 
employing units. 1 

Pennsylvania's law provides that "disclosure of records or information may be authorized by 
specific instruction of the Department or Board in the following cases: ... (7) To individuals, 
organizations and agencies, or for purposes other man as specified in paragraphs (1H6) of this 
section, if such a disclosure shall not impede the operation of, and is not inconsistent with 
the purposes of the employment security program {emphasis added], and is authorized in 
writing in individual cases by the Executive Director." 22 This language recognizes that expUdt 
legislative stipulations may not cover specific requests for access to administrative records; 
requests that are consistent with an Executive Director's custodial interpretation of legislative 
inten», and individual and organizational passive consent. 

West Virginia's law provides that "-.the commissioner may provide information thus obtained 
to the following governmental entities for purposes consistent with state and federal laws: ... (4) 
Those agencies of state government responsible for economic and community development; 
secondary, post-secondary and vocational education; vocational rehabilitation, employment 
and training, including, but not limited to, the administration of the perkins act and the Job 
training and Isic] partnership act; 

Excerpts from 17 State laws have been presented in the first part of this Appendix. They are 
examples of the confidentiality stipulations that were found in a review of the 50 State statutes 
olus the District of Columbia. 

In all cases, the central issue is whether transfer of administrative records to an external party 
constitutes release to the public; and, if so, whether the presence of a unique identifier (e.g. a 
social security number) on a record may reveal an individual's identity, or creates a presumption 
that the identity may readily be ascertained. 

In the next part of this Appendix, these State laws are complemented by an examination of 
administrative documents that were submitted by 26 / the SESA administrators. These 
documents reflect the administrators' attempts to trans! „■ law into everyday practice. 

Administrative Documents 

California's Employment Development Department Information Security Officer states: "There 
are two broad categories of issues th<*t arise under the confidentiality provisions of the 
Unemployment Insurance Code. The first is concerned with the conflict between the law and 
the necessity or desirability of releasing information which is confidential under the 
Unemployment Insurance Code io an entity outside the Employment Development 
Department, There is no body of litigation relating to this particular issue. However, there is 
case law relating to the second issue, which involves third parties seeking to have the 
Department disclose confidential information pursuant to judicial and administrative 
proceedings. These are generally termed discovery proceedings.' 
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Colorado's Department of Labor and Employment provided the author with a copy of the 
Information and Instructions Manual for their CA-TOP SECRET Security System software 
package that is resident on the State's mainframe computer. This was complemented by She 
Department's implementation package for third-party access to the Department's computer 
systems. 

Delaware' s Department of Labor provided a one-page Vendor Agreement of Non-Disclosure 
form, which documents the purpose and manner of intended use and offers assurance that the 
Department's own non-disclosure requirements are met. 

Florida's Department of Labor and Employment Security provided a memorandum authored 
by their General Counsel, which slates mat: "Any confidential information so released [to public 
employees in the performance of their public duties] should continue to be held confidential 
and not be subject to further rel«>se In fact, it may be necessary to obtain an agreement to this 
effect...lf any information is released the person or entity that it is released to should agree to 
abide by the confidentiality provisions [of the Department], and be made aware of the penalties 
for release of information ... 

Waho's Department of Employment provided a copy of the section of their Administrative 
Procedure Manual that covers disclosure of information. This document spells out authorities, 
and requirements for written agreements; and summarizes current disclosure of information 
agreements that have been approved by the Department. 27 

Illinois' Department of Employment Security provided the Final Report and Recommendations 
of the Sub-Committees of the IDES Data Release Policy Committee, and related procedural 
documents. 

Indiana's Department of Employment and Training Services provided a communication 
memorandum covering contracting procedures for the release of confidential information from 
the Department's automated records. 

Michigan's Employment Security Commission provided a three-page matrix that describes 
their policies for release of Income and Eligibility Verification System Information? 0 

Minnesota's Departr. ent of Jobs and Training provided a chapter from Minnesota's 
Government Data Practices Act, which is intended to serve as a guide to agency employees in 
responding to requests for the release of administrative records. 31 This document contains 
definitions that appear to be inconsistent with those adopted elsewhere {e.g., in Reynolds' paper 
and in The Common Federal Policy on the Participation of Human Subjects in Research). Here, 
confidential data is defined as "not-public data which isn't accessible to the individual data 
subject", and private data is defined as "data which is not-public and is accessible to the 
individual data subject". Not-public data, in turn, is defined as "government data which is 
classified by statute, federal law, or temporary classification as confidential, private, nonpublic, 
or protected nonpublic". 

Mississippi's Employment Security Commission provided an Executive Bulletin that covers 
information disclosures This Bulletin contains examples of requests for data and suggested 
responses. Among the examples is: "J. Any information request, not already covered 
hereinabove, which is made by a public employee in the performance of his public duties may 
be acceded to under the following circumstances: (a) The request is for specific information, and 
is wntten on official letter-head stationery; (b) States the reason the information is needed; and 
(c) That such necessity arises from the requesting individual's performance of his public duties. 
If these criteria are met the information may be furnished by transmittal letter which states that 
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the described information is transmitted pursuant to the letter request, and that the information 
is delivered for the purposes there stated only, and will otherwise be held in strict confidence 
by the recipient" 35 

Nebraska's Department of Labor provided the portion of its Policy Manual mat addresses 
protection of confidential information, and a five-page Nebraska Sample Agreement form that 
is used for one-year renewable approval for the transmittal of administrative records. 

North Carolina's Employment Security Commission provided its policy document mat covers 
disclosure of information. 35 Included in this policy statement is the Chief Counsel's official 
opinion that "the Federal Privacy Act does not apply to the Employment Security Commis^n 
of North Carolina. That Act applies only to federal agencies. No infoirmuonshall be released 
lor withheld?! because of any provisions contained in the Federal Privacy Act This document 
also states that It is the Policy of the Employment Security Commission of Norm Carolina to 
permit disclosure of information from the files and records of the Employment Security 
Commission of North Carolina: ... C To any officer or employee (including law enforcement 
offices) of any agency of the federal government or of a state or territorial govei nment, lawfully 
charged with the administration of a federal, state, or territorial law, but only for purposes 
reasonably necessary or the proper administration of such law and only after receiving a written 
request for information. 7 

Oregon's Employment Division provided a copy of its Confidentiality Handbook, 38 which 
includes an Application for Confidential Information form. 

Tennessee's Department of Employment Security provided several Attorney General opinions 
and an Unemployment Insurance Directive addressing disclosure of information issues. This 
Directive is intended to serve the needs of field office personnel, and does not offer new 
information of use here. 
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CHAPTER IV 
A Study Of The Use Of 
Unemployment Insurance 
Wage-Record Data As An Evaluation 
Tool for JTPA 



Introduction 



In May of 1988 the National Commission for Employment Policy (NCEP) launched a multi-state, 
multi-year project to examine the potential for using Unemployment Insurance (UI) 
wage-reco-d data as an evaluation tool for programs funded under the Job Training Partnership 
Act (JTPA). The State of Illinois served as the clearinghouse for the data collection effort This 
chapter presents the findings from Phase II of the project. 

Two broad analytical goals have guided this study: 

• To compare the results of the proposed UI wage-record approach to the results of the 
current telephone survey approach used by the U.S. Department of Labor (DOL) to 
measure posi-program outcomes under JTPA; 

• To describe the pre-program and post-program employment and earnings experiences 
of JTPA participants, by demographic subgroups of participants and types of services 
provided. 

Achievement of these goals depended on the successful development of a multi-state database 
which integrated UI and JTPA information. Thus, a major focus of Phase I was to demonstrate 
the technical feasibility of Unking records from the State JTPA and UI systems. Eleven States 
pined the NCEP in this effort. The study showed that the technical problems of linking records 
from the two data systems were relatively minor and easily resolved. Even those States that had 
little or no experience with integrating JTPA and UI data were successful in establishing and 
executing the necessary agreements and procedures for exchanging data. 

Phase II of the project is both an extension and expansion of the first phase. In Phase II, four 
new States were added to the list of already participating States - Georgia, Kentucky, Maryland, 
and Texas. In addition, the database was expanded to include participants who left the program 
during Program Year 1987 (PY87). Finally, for the participants who left the program during 
Program Year 1986 (PY86), four more quarters of UI data were obtained. 2 

With regard to technical feasibility, the broad findings and conclusions from Phase II were the 
same as those from Phase I: in summary, participating States were able to accomplish the 
required data linkages. The Phase II effort revealed that the problems which States experienced 
in generating the JTPA and UI data files were easily corrected by the States once the problems 
were identified. In most cases, the problems stemmed from errors in the computer programs 
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used to extract the data from the master files. Appendix IV.A contains a discussion of the 
development of the Phase II database, including new problems and the procedures used to test 
the database for completeness. 

The remainder of this chapter focuses on the substantive analyses performed during Phase U. 
It is organized into two major sections. The next section presents analyses designed to assess 
the quality of the telephone survey data and the UI wage-record data. Special attention is given 
to those aspects of the UI data that might hinder their use for program evaluation and 
performance management. In the subsequent section, UI wage-record data are used to compare 
pre-program and post-program employment and earnings trends across program years andby 
various subgroups in the JTPA Title II-A adult population. This includes an exploratory effort 
to track the "flow" of clients into and out of employment over the course of the post-program 
period. 

Post-Program Survey Data And UI Wage Data 

The 11 -State database constructed for Phase 1 provided an opportunity to examine several key 
issues surrounding the quality of the post-program survey data. The previous study revealed 
that certain subgroups of the JTPA population had lower response rates even after controlling 
for the effects of employment status at termination. Since Service Delivery Areas (SDAs) serve 
different mixes of these subgroups, this finding explains part of the variation in the response 
rates reported by SDAs. 

In addition, the evidence suggested that those participants who responded to the telephone 
survey had different post-program experiences than those who did rot respond. The 
consequence of these differences is that estimates of program performance derived from the 
survey tend to overstate the actual levels of performance. 

Finally, the Phase I study found a fairly high degree of correspondence between employment 
information obtained from respondents to the survey and their UI wage records. Although data 
limitations inhibited efforts to isolate the source of me differences that did emerge, the evidence 
suggested that there were two major explanations for these discrepancies: (1 ) self-employment, 
which is not covered under the UI system, and (2) out-of-state employment, which is recorded 
in a different State's UI system. In both cases, remedial measures to reduce the effect of these 
UI coverage issues were suggested. 

In this section, the analyses presented in the Phase 1 report are replicated using the expanded 
Phase II database. This replication tests the generalizability of the Phase I findings. The addition 
of four new States allows us to determine thedegree to which the Phase I findings were a function 
of the particular set of 11 States used in the earlier analyses. If the same results are obtained with 
the 15-State database, this would lend support to the view that the findings from the project are 
an accurate reflection of the national experience. 

Further, the addition of PY87 information to the database permits an assessment of the stability 
of the findings over time. The additional year's experience in conducting post-program surveys 
may have resulted in more effective data collection strategies and techniques. If so, the Phase I 
findings might not reflect the effectiveness of current data collection efforts. 

NONRESPONSE BIAS 

Across the 15 States, 63,085 PY86 adult terminees were selected for the post-program telephone 
survey; and 41,770 responded to all three mandatory questions, producing an overall response 
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rate of 66.1%. In PY87, 42,987 of the 61,751 adult terminees selected for the survey responded 
to all three mandatory questions, producing an overall response rate of 71 .2%. 4 

These results indicate that the States were more adept at contacting their former clients in PY87 
than they were in PY86. In fact, the overall PY87 response rate of 71 .2% was above the DOL's 
requirement that each SDA should achieve a minimum response rate of 70%. Despite this overall 
improvement, many SDAs (and States in aggregate) were still experiencing difficulties in 
achieving the required minimum response rates; over 40% of all SDAs in the nation had response 
rates below the 70% minimum in PY87. However, there have been marked improvements since 
PY87. Recent estimates by the DOL indicate that only 20% of all SDAs had response rates below 
70%inPY90. 

While the overall response rate improved from PY86 to PY87, certain subgroups of the JTPA 
population are still more likely than others to respond to the post-program survey (see Table 1). 
The greatest difference is found among the two subgroups denned on the basis of their 
employment status at termination. Clients who are employed at termination from the program 
have response rates that are over 15 percentage points higher than clients who are unemployed 
when they leave the program. In addition, some differences appear across sex, race, education, 
and welfare subgroups. While some of these differences are small, they are persistent across the 
two program years. 



TABLE 1 

POST-PROGRAM SURVEY RESPONSE RATES BY CLIENT SUBGROUPS 
PY86 and PY87 Title II-A Adult Survey Populations: Phase if 



Participant Characteristic 


Response Rate FY 86 


Response Rate FY 87 


1. Sex 






Males 


63.8 


68.6 


Females 


67.8 


72.9 


2. Race/Ethnicity 






White 


675 


723 


Black 


653 


70.2 


Hispanic 


61.4 


707 


American Native 


51.9 


56.6 


Asian or Pacific Islander 


66.1 


662 


3. Educational Status 






High School Dropout 


625 


67.7 


High School Graduate 


672 


72& 


Post High School Education 


69.0 


72.9 


4. Welfare Status 






Non Recipient 


66.6 


713 


AFDC Recipient 


65.8 


71.7 


Genera] Assistance 


52.6 


51.8 


Refugee Assistance 


655 


65.7 


5. Employment Sums at Termination 






Employed 


71.4 


75.6 


Unemployed 


52.7 


582 


Overall Response Rate 


€62 


712 



'Phaae II States include FU CA, ID, IU IN, MO, NV, OR, SC TK, UT, VA, and WA. 
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It is difficult to draw any firm conclusions regarding the subgroup differences displayed .n Table 
1 because the observed differences in response rates may be related to some underlying 
difference between the groups being compared. For example, since the demographic subgroups 
tend to have different levels of employment at termination from the JTPA program, it is possible 
that their differences in response rates are merely reflecting their different entered -employment 
rates. 

To address mis issue, a multivariate analysis was used to estimate the independent effects of 
employment status at termination and subgroup membership on the probability of responding 
to the survey. Since the States exhibited different overall response rates to the survey, "dummy 
variables' 1 were included in the regression model to account for variations in the effectiveness 
of the States' data collection efforts. 

This analysis indicated that some of the subgroup differences shown in Table 1 were not 
statistically significant after accounting for the effects of other factors that might influence 
responses to the survey (see Table 2>. Males, blacks, high school dropouts, and those 
unemployed at termination were all less likely to respond to the survey in both program years. 
However, the lower response rates for Hispanics and welfare recipients can be attributed to 
sources other than membership in these subgroups. Indeed, Hispanics were found to be more 
likely to respond to the survey in PY87 after controlling for the effects of other factors. 



TABLE 2 

REGRESSION RESULTS FOR A MODEL PREDICTING RESPONSE TO 

THE POST-PROGRAM SURVEY 
PY86 and PY87 Title II-A Adult Survey Populations 

Phase II States 3 



Variable 


Regression Coefficient PY86 


Regression Coefficient PY87 


1. Employed At Termination 


.198 


.177" 


2. Male 


-.049 


"W" - 1 " 

-.052 

— 


3. Black 


-.040 


-.019 


4. Hispanic 


.009 


a 

.023 


5. High School Dropout 


-.048" 


-.045" 


6. Welfare Recipient 


.002 


.002 




^=.09 


R 2 - .07 




N = 63,086 


N = 61,752 



Significance Levels: * = alpha .01 
" -alpha .001 



* Phase Ii States Include: FL, CA, ID, 1U IN, MO, NV, OR, SC, TX, UT, VA, and WA. 



These findings differ from those in the Phase I study in an important respect In the earlier 
analysis, both Hispanics and welfare recipients were found to be less likely to respond to the 
post-program survey. The reason for the conflicting findings became apparent when the present 
analysis was repeated using only the original 11 participating States (i.e., by deleting the new 
Phase II States). Once again, lower response rates were found for both Hispanics and welfare 
recipients (see Table 3). 
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The differences introduced by including the new States imply that there are material variations 
among States in their ability to contact certain subgroups of JTPA terminees/ If this were not 
the case, then the response rate results should be fairly robust to the introduction (or deletion) 
of States to (or from) the database. In conclusion, the differences for Hispanics and welfare 
recipients indicate that the effectiveness of post-program data collection varies across the States, 



TABLE 3 

REGRESSION RESULTS FOR A MODEL PREDICTING RESPONSE TO 

THE POST-PROGRAM SURVEY 
PY86 and PY87 Title II-A Adult Survey Populations 
Phase I States Only 3 



Variable 


Regression Coefficient FY 86 


Regression Coefficient FY 87 


1 . Employed At Termination 


.219" 


.191" 


2. Male 


-.066*" 


-.075" 


3. Black 


-.055" 


-.043" 


4. Hispanic 


-.049"" 


-.029"" 


5. High School Dropout 


-.050" 


-.038" 


6. Welfare Recipient 


-.015" 


-.019" 




R 2 = .09 


R 2 = .08 




N = 45,953 


N = 42,111 



Significance Levels: * = alpha .01 
" =alpha 001 



* Phase I States include: R, ID, 1L, IN, MO, NV, OR,SC,UT,VA, and WA. 



Further comparisons using Tables 2 and 3 illustrate that the results are fairly stable over time. 
In each of these tables, the corresponding parameter estimates for each subgroup are roughly 
the same across the two program years. Thus, while States became more proficient in collecting 
the follow-up data (i scall Table 1), they were not successful in decreasing the response rate 
differentials among su bgroups of JTPA terminees. 

The US. Department of ) -abor (DOL) has developed a statistical adjustment procedure to reduce 
the nonresponse biases Implied by the above response rate differences. The DOL requires SDAs 
to use this procedure when the response rate for their clients who are employed at termination 
differs from the response rate of those unemployed at termination by more than 5 percentage 
points. Since employment status at termination is the most influential factor predicting the 
probability that an individual will respond to the survey, the adjustment procedure addresses 
the major source of bias in the survey-based estimates. 

However, DOL's adjustment procedure does not address all the possible sourcesof nonresponse 
bias. For example, the results of the above analysis indicate that blacks are less likely to respond 
to the post-program survey, after accounting for their employment status at termination. Since 
the adjustment procedure does not address such response rate differences, the estimates of 
post-program performance based on the survey are likely to be biased. 

Moreover, the adjustment procedure assumes that respondents have the same experiences as 
non-respondents within each of the two employment categories. For example, it assumes that 
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the post-program experiences of respondents who were employed at termination are 
representative of the experiences of nonrespondents who were employed at termination, rae 
effectiveness of the adjustment procedure is a function of the degree to which thisassumption 
is valid. If it is violated, there are even circumstances in which the adjustment procedure could 
increase the bias. 

In the Phase I study, the UI wage-record data revealed that respondents and nonrespondentsto 
the telephone survey had different post-program experiences. That analysis suggested that 
these differences had the potential of introducing considerable bias in evaluating JTPA s 
performance from the survey results. 

Replication of the Phase I analysis using the expanded database from Phase II reinforces the 
earlier findings and conclusions. The employment rates based on Ul data were very different 
for the respondent and non-respondent groups. While 75% of the PY87 survey respondents 
were reported as employed (i.e., had positive UI wages) for the first full post-program quar**, 
only 63% of the nonrespondents were reported as employed. This indicates that estimates of 
post-program employment based on the self-reported experiences of survey respondents may 
overstate the actual level of employment among terminees. 

Further sources of nonresponse bias are illustrated in Table 4, which displays the UI-based 
employment rates for PY87 survey respondents and nonrespondents classified by their 
employment status at termination. Those employed at termination were more likely to respond 
to the survey (75.1% compared to 57.1%) and to be employed during the first post-program 
quarter (80.7% compared to 40.0%) than those unemployed at termination. While (the 
post-program experiences of respondents and nonrespondents were similar for those 
unemployed at termination (39.6% compared to 40.6%), there is a marked difference in the 
post-program employment status of respondents and nonrespondents for those employed at 
termination (523% compared to 75.3%). 

The last finding has important implications. Within the group of terminees who were employed 
at termination, the subgroup of respondents had higher rates of post-program employment than 
the subgroup of nonrespondents. This indicates that the subgroup of respondents is not a 
random subset of the group of terminees who were employed at termination. DOL's adjustment 
procedure addresses the potential bias produced by differences in response rates between the 
employed and unemployed groups. However, it does not address the potential bias resulting 
from the different post-program experiences of the subgroups of respondents and 
nonrespondents within the two groups. Within the group of employed terminees, the subgroup 
of respondents had a higher post-program employment rate than the subgroup of 
nonrespondents; in this case, the survey provides inflated estimates of JTPA participants' 
post-program employment rates and therefore the program's performance. 

A cell by cell comparison of Tables 4 and 5 reveals a remarkable similarity in the UI-based 
employment rates between PY87 and PY86. This suggests that the bias in the survey data is 
roughly constant across program years. If so, the post-program employment rates obtained 
from the survey would provide accurate estimates of changes in JTPA's performance from year 
to year, even though each annual estimate of performance is likely to be upwardly biased. 




J 



•A 



138 



TABLE 4 

COMPARISON OF THE EMPLOYMENT RATE FOR THE FIRST FULL 
QUARTER AFTER PROGRAM TERMINATION FOR RESPONDENTS AND 
NONRESPONDENTS BY EMPLOYMENT STATUS AT TERMINATION* 
PY87 Title II-A Adult Survey Population 
Phase II 





Employment Status at Termination 


Post-Program Survey Status 


Employed 


Unemployed 


Respondents 


825 


39.6 


Nonrespondents 


753 


40.6 




Response Rate 


75.1 


57.1 


First Quarter Employment Rare 


80.7 


40.0 



* Employment rates calculated from Ul wage-record data. 



TABLE 5 

COMPARISON OF THE EMPLOYMENT RATE FOR THE FIRST FULL 
QUARTER AFTER PROGRAM TERMINATION FOR RESPONDENTS AND 
NONRESPONDENTS BY EMPLOYMENT STATUS AT TERMINATION* 
PY86 Title II-A Adult Survey Population 
Phase II 





Employment Status at Termination 


Post-Program Survey Status 


Employed 


Unemployed 


Respondents 


82.7 


39.8 


Nonrespondents 


752 


39.7 




Response Rate 


7UD 


51.8 


First Quarter Employment Rate 


805 


39.8 



1 Employment rates calculated from Ul wage-record data 



A further analysis was designed to assess the effect of this survey Has on the estimates of SPAS' 
performance in PY87. For each SDA, the Ul-based employment rate was calculated for (1) the 
entire survey sample, respondents and nonrespondents together, and (2) respondents alone. 
While the correlation between the two rates was high (r=,90), there were some substantial 
differences between the two rates for individual SDAs. Even after DOL's agistment procedure 
was applied, the standard deviation of the difference between the two rates was more than 2 
percentage points. This suggests that the bias in the survey-based rates may have a substantial 
effect on the interpretation of an individual SDA's performance. 
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COMPARISON OF UI WAGE DATA TO JTPA OUTCOME DATA 



In the Phase I study, the telephone survey and UI data were compared for the subset of survey 
respondents whose 13-week post-program period coincided exactly witha calendar quarter. In 
that analysis, the responses of the survey participants were examined to determine whether they 
claimed to have been employed at any point during the 13-week post-program period. This 
self-reported employment status was then compared with the Ul-based employment status in 
order to assess the degree of correspondence between the two data sources. 

This exercise was repeated using the Phase U data. Of the PY87 survey respondents, 79.6% were 
placed in the same employment categories by the two data sources, which is only slightly lower 
than the 81.7% found for the PY86 respondents. However, unlike the earlier analysis, which 
found that the bulk of discrepancies were the Survey-employed / Ul-unemployed case as 
opposed to the Survey-unemployed/ Ul-employed case (132% and 5.1%, respectively); the 
present analysis found the discrepancies to be evenly split between the two cases (9.1% and 
1 1 3%, respectively; see Table 6). 

An examination of the 239 respondents in P V87 who claimed to be unemployed during the entire 
post-program period, but for whora UI wages were found, revealed that 184 (77%) were reported 
by the JTPA system as being employed at termination from the program. While some of these 
respondents may have lost their jobs between their termination date and the onset of their 
13-week post-program period (less than one week in all cases), most probably did not Hence, 
it is reasonable to conclude that memory problems play a major role in explaining the 
discrepancies for the Survey-unemployed /Ul-employed case. 



TABLE 6 

COMPARISON OF THE EMPLOYMENT STATUS RESULTS FOR THE 
13TH-WEEK POST-PROGRAM SURVEY AND UI WAGE-RECORD DATA 

PY87 Title II-A Adult Survey Population 





First Quarter UI Status 


Employment Status Fez Survey 


Employed 


Unemployed 


Totals 


Employed 


1414 

(66.7%) 


193 

(9.1%) 


1607 
(75.8%) 


Unemployed 


239 
(113%) 


274 
(12.9%) 


513 
(242%) 


Totals 


1653 
(78.0%) 


467 

(22.0%) 


2120 
(100.0%) 


Overall Correspondence - 1688 (79.6%) 



There are three major explanations for the findings in the Survey -employed / Ul-unemploy ed 
case. A JTPA participant may have been (1) employed out-of-state, (2) employed in a job not 
covered under the UI system, or (3) employed in the quarter but wages were not paid by the 
employer until the next quarter. 8 

To assess the relative influence of these factors, the clearinghouse obtained PY87 data from the 
State of Illinois describing the employment status of adult participants ?t termination. These 
data were recorded on the State's JTPA management information system (MIS). For each 
participant, the employment status indicated on their MIS record was checked against the 
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employment status indicated on their UI wage record. The mismaivhes were then examined in 
detail to identify the reason for the conflict. 

Use of the MIS termination data, in lieu of the post-program survey data, has two major 
advantages for present purposes. First, these data are not subject to the nonresponse biases 
affecting the survey data. Second, the number of observations is much greater. Unlike the 
survey data which are only a sample from the universe of program terminees, the MIS data 
represent the universe. 

For the State of Illinois, the clearinghouse had access to the employer's name reported on the 
MIS as well as the employer's "doing-business-as" name reported on the UI system. These two 
pieces of information were the lay to examining the UI coverage and reporting issues. 

Table 7 demonstrates the results of matching the PY87 MIS termination information with the UI 
wage-record information reported in the quarter of termination. The overall correspondence 
between the UI and MIS data (73.9%) was somewhat lower than the correspondence between 
the UI and survey data (79.6%) shown in Table 6. Although the percentage of mismatches 
between these data sources is greater than between the UI and survey data, the distribution of 
mismatches is more evenly split between the case of those who were employed according to the 
UI wage records but not employed according to the MB, and the reverse case. 

The MIS data permitted a detailed examination into the reasons why an MIS-reported condition 
of employment could go uncorroborated by the UI data system. Since the MIS record contained 
the address of the employer, it was possible to determine if the participant was employe : 
out-of-state. As shown in Table 8, the total percentage of out-of-state employment for this grouj. 
of uncorroborated cases was 153% (only 2% of all terminees). Further examination revealed 
that the majority of these cases occurred in two SDAs that bordered a large urban area located 
in another State. 

TABLE 7 

COMPARISON OF THE EMPLOYMENT STATUS AT TERMINATION AND 

UI WAGE-RECORD DATA 
Illinois PY87 Title II-A Adult Population 





Termination Quarter UI Status 


Employment Status at Termination 


Employed 


Unemployed 


Totels 


Employed 


13758 
(532%) 


3387 
(13.1%) 


17145 
(663%) 


Unemployed 


3367 
(13.0%) 


5347 
(20.7%) 


8714 

(33.7%) 


Totals 


17125 
(662%) 


8734 

(33.8%) 


25859 
(100.0%) 


Overall Correspondence » 19105 (73.9%) 
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TABLE 8 
CAUSES FOR MISMATCHES: 
PARTICIPANTS EMPLOYED AT TERMINATION WITH NO UI RECORD 
FOR THE QUARTER OF TERMINATION 
Illinois PYB7 Title II A Adult Population 



Reason for Mismatch 


f of Cases 




1. Employed out of State 


517 


(153%) 


2. Sdf-Emptoved 


51 


(13%) 


3. Federal Employment 


172 


(5.1%) 


4. Within-ProfEram UI Record 


81 


(2.4%) 


5. 1st Ouarter Post UI Record 


608 


(18.0%) 


6. 2nd Ouarter Post UI Record 


93 


(2.7%) 


7. No Related UI Record 


1865 


(55.0%) 


a) No UI Record 


1325 


(39.1%) 


b)Mismatched Employers 


540 


(15.9%) 


Total 


3387 


(100.0%) 



Next the possibility that a participant had a job at termination which was not covered under 
the UI system, such as self-employment, was considered. Although the Illinois MIS does not 
have a separate data field for reporting self-employment, the study attempted to infer 
self-employment from other data items that are available. For example, the employer's name 
field on the MIS w*s scanned to determine if self-employment was reported. The tests included 
a search of the en plover's name for the character string "self and a comparison of the employer' s 
name and phone number with the participant's name and phone number. The results of these 
tests (see Table 8) suggest that few JTPA participants fell into the self-employed category (15%). 

Forms of covered employment for which wage records are not available include employment 
by the federal government (including military), railroad workers, and small nonprofit 
organizations. By visually checking the employer's name given on the JTPA MB, it was 
determined that approximately 5.1 % of all uncorroborated MIS employment could be attributed 
to these other forms of employment, most commonly, federal employment. Combined with the 
previous estimate of self employment, such employment accounted for less than 7% of the 
conflict between me two data sources. 

The third possible reason for not finding UI wages for an "employed" JTPA terminee is that the 
person may have worked during the quarter but wages were not paid until the following quarter. 
This type of mismatch was examined by comparing the employer's name reported in the 
termination record to the "doing-business-as" name reported in the UI database. The analysis 
included UI records for the participant's within-program quarters (i.e., all quarters within which 
a participant was still active in JTPA for at least a day) along with records for the first two 
post-program quarters. 

Table 8 displays the percentages for the three types of employer name matches. The first type 
of match occurred when an employer's name recorded on the MIS agreed with the 
"doing-business-as" name recorded on a UI record for one of the within-program quarters. This 
match was examined because there could be errors in recording the termination date of the 
participant. In such cases, there would be no UI wages reported by the employer specified on 
the MIS record for the quarter of termination, but the participant would have UI wages reported 
for a within-program quarter. 
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The other two types of matches occurred when the employer's name recorded at termination 
agreed with the employer's name recorded on a UI record either one or two quarters after 
termination. These cases would occur if a participant worked during the quarter of termination 
but received their pay at a later point in time. 

As seen in Table 8, over 23% of uncorroborated MIS employment could be attributed to either 
late MIS reporting or the "earn/pay" lag. The latter is the major explanation: the percentage of 
cases where the employer's name matched in the first quarter after termination was 18%. The 
percentage of matches round during the within-program and second post-program quarters 
were a much lower 2.4% and 2.7%, respectively. 

Several important conclusions may be drawn from this analysis In Illinois, out-of-state 
employment at the time of termination is infrequent for most SDAs. (As discussed in Chapter 
II, this holds for the nation as a whole.) Similar results would be expected during the 
post-program period. However, since out-of-state employment is high for some SDAs, it should 
be taken into account in JTPA's system of performance management One approach would be 
to use a statistical adjustment procedure, akin to the nonresponse adjustment procedure used 
with the current telephone survey. A more direct approach would be to develop interstate data 
sharing agreements. 

The most frequent reason for an uncorroborater 1 MIS employment is the earn/pay reporting lag. 
Although the number of participant records affected by the lag is somewhat large, the overall 
effect of the earn/pay lag on estimates of post-program employment may be small. This is 
because the number of people who worked during a given quarter *T* but did not receive their 
wages until quarter T+F will be partially offset by the number of people who lost their job in 
quarter T-l" but had wages reported for quarter T*. Such offsetting of errors improves the 
accuracy of estimating post-program employment rates. 

Of greater concern is the large number of cases for which an MIS-reported employment could 
not be corroborated through UI records. A total of 55.0% of the cases with a mismatch contained 
MIS employment information which did not correspond to any available UI wage record. One 
possible explanation is that in order for an MIS employer name to be declared a match with the 
UI "doing business as" name, the names had to be nearly identical. In some cases (15.9%), a UI 
record was found during the first or second full quarter after termination, but the employer 
names did not meet the stringent criteria used to determine a match. It is not known how much 
of this Ul-measured employment is associated with the participant's employment at 
termination, as recorded on the MIS, but it may account for a significant number of the 
non-matches. 

Putting aside the preceding mismatches, 39.1% of the MIS-employed/UI-unemployed cases 
remain to be explained. An examination of the employer's name reported on the JTPA MIS for 
these 1,325 participants revealed 1,108 different employers. Furthermore, while it was expected 
that a large percentage of these employers would be employers that were exempt from UI 
reporting, only 5.7% of the employers appeared to faU into this category/ The rest were 
companies and businesses with no apparent reason for being exempt from UI reporting. 

Possible explanations for the inability to match these remaining cases include (1) errors in 
reporting the social security number on the JTPA or UI data systems, (2) an employer's neglect 
of UI reporting requirements, and (3) recording errors by JTPA operators. Given that many of 
these employers are highly established companies and that the uncorroborated employment 
appears to be randomly distributed across SDAs, the most likely explanation is errors in 
recording social security numbers. 



This analysis of cases where a UI wage-record could not be linked to an MIS-reported 
employment has illustrated some of the problems that may arise with the use of UI date. 
However, there is a similar number of cases where a participant earned UI wages during the 
termination quarter, but the participant was reported as unemployed at termination on the 
State's MIS. 

There are a number of valid reasons why a participant can have UI wages reported fothefr 
quarter of termination, yet not be reported as employed at termination. Many of these 
explanations involve the timing of employment For example, a JTPA participant could become 
employed during the calendar quarter of termination but after the date the person leaves the 
program. This is more likely if the date of termination occurs early in the quarter than if it occurs 
late. Or, the person could have worked in the previous quarter but have been paid during the 
termination quarter. 

It is also plausible that the participant was employed on the termination date but the SDA was 
unaware of the employment. For example, the SDA could have lost track of the participant and 
administratively purged the person from its files as unemployed at termination. 

Further analysis suggested that some SDAs were having more difficulty than others keeping in 
contact with their participants. This was indicated by the numbers of PY86 and PY87 terminees 
whom sbAs reported as unemployed at termination but the UI system recorded as employed 
during the quarter of termination. The results showed a wide variation in the percentages 
across the SDAs, from less than 1 % in some cases to greater than 20% in others (average=10.2%). 

The finding that some SDAs keep track of their clients better than others is not surprising. 
However, the strength of the relationship between "missed employment" and the entered 
employment r^teat termination is startling: the correlation is -.68! This suggests that the entered 
employment rate at termination calculated from MIS data depends strongly on the effectiveness 
of an SDA's tracking and reporting system If an SDA is having great difficulty keeping In 
contact with its participants, it may report an entered employment rate appreciably different 
from its true one. 

Employment And Earnings Trends 

The Phase I report contained a series of graphs which displayed the pre-program and 
post-program employment and earning trends for various subgroups of the JTPA population. 
This section reproduces those graphs using the expanded Phase II database. The new analyses 
illustrate the usefulness of wage-record data for tracking the post-program experiences of JTPA 
participants over an extended period of time. 

The Phase II database makes it possible to observe the post-program employment and earnings 
trends of PY86 terminees for an additional four quarters. It is also possible to compare the 
short-term trends of PY86 terminees with their PY87 counterparts to assess whether there are 
any significant differences in the labor market experiences of the two cohorts. 

Prior to examining the employment and earnings trends of former JTPA participants, a 
cautionary note is offered. These trends are presented solely to illustrate the potential of UI 
information for conducting longitudinal analysis. Conclusions about the program's 
effectiveness should not be drawn from these trends, since they are not adjusted for human 
capital and other individual differences affecting labor market outcomes. Conclusions about 
the net impact of the JTPA program cannot be drawn from the employment and earnings 
trends displayed in this chapter. 
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Studies designed to assess the net impact of job training services generally use a comparison 
group (ideally a pure control group) of non-participants to approximate what the post-program 
labor force experiences of participants would have been if they had not participated in the 
program. The net impact of the program is measured by the difference between these 
hypothetical outcomes and the actual ones for the participants involved. Currently, the NCEP 
is sponsoring a separate study in which UI data from the State of Utah are being used to explore 
methodological issues involved in estimating the net impact of JTPA's Title II- A program. 12 A 
comparison group is being drawn from the population of Utah's job service registrants. The 
selected individuals have demographic characteristics and labor force experiences that are 
similar to the State's JTPA participants. 

The present study does not pursue a net impact approach. Rather, it documents prevalent 
patterns in the labor market experiences of JTPA participants before and after they receive their 
training services. Identification of these patterns is a useful first step in evaluating program 
effects. Net impact studies are a next step. 13 

EMPLOYMENT TRENDS 

The graphs depicting the employment trends for PY86 and PY87 participants by various 
subgrorps are in Appendix IV.B (see Figures B.l - B\5). In general the proportion of JTPA 
termint.es who are employed (i.e., earn some Ul wages during a designated quarter) declines 
throughout the post-program period, with the greatest decline found between the first and 
second quarter after leaving the program. After the second quarter, the rate of decline appears 
to be fairly constant throughout the remainder of the post-program period. In addition, the 
graphs illustrate that the trend for the first post-program year for the VY87 terminees is similar 
to that for the PY86 terminees. 

One major difference between the Phase I and Phase II results involves the post-program 
employment pattern for participants who received remedial/basic classroom training. 
According to the Phase I data, employment rates for such participants rose after the second 
post-program quarter. It was suggested in the Phase 1 report that this rise could be the result of 
a delayed effect of remedial/basic classroom training. 

However, an examination of the employment trend for this type of training in the Phase II 
database reveals that employment rates consistently decline throughout the post-program 
period, much like the other types of training (see Figure B.5). Furthermore, tliis decline is found 
among both PY86 and PY87 terminees. In conclusion, there is no longer support for the earlier 
suggestion that remedial/basic classroom training has de ayed effects on the probability of 
employment. 

This difference between the Phase 1 and Phase II findings was caused by two factors. The first 
was the resolution of the problems that a few States were experiencing in extracting information 
from their UI data systems. It has already been noted that correction of these problems 
eliminated the relatively large third-to-fourth quarter employment decline noted in the Phase I 
data (see Figure 1 in Appendix IV A). The second contributing factor was the small number of 
cases in which a JTPA client received only remedial /basic education. In PY86, less than 2,000 
of the 63,086 terminees received remedial/basic classroom training as their only intervention. 
As a result, it did not take many updates in the UI data to produce noticeable changes in the 
employment rates. 
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EARNINGS TRENDS 



The average earnings of employed terminees show a steady increase throughout the 
post-program period. Like the employment rates, the earnings patterns are remarkably similar 
across the two program years. The differences that do emerge for certain subgroups, e.g., the 
relatively high level of earnings for PY86 terminees who received refugee assistance, can be 
attributed to the small number of individuals in those subgroups. 

One interesting finding is that the male/female earnings differential increases over the course 
of the post-program period. Specifically, the earnings differential increased from about $500 in 
the first post-program quarter to about $750 in the eighth post-program quarter. By way of 
possible explanation, the male/female earnings differential decreased during the pre-program 
period, so that the final gap (post-quarter 8) is similar to the initial gap (pro-quarter 4). These 
results suggest that the males may have experienced a more severe disruption in their earnings 
prior to program participation titan the females and therefore a slower recovery afterwards. If 
so, the growing differential in the post-program period may be unrelated to the training services. 

On a broader methodological note, Figure B.6 depicts a series of cross-sectional snapshots of 
terminees who are employed in a given quarter, not a longitudinal analysis of the earnings 
experiences of a fixed cohort of terminees. For this reason and others given earlier, such earnings 
trends do not represent the net impact of program services for different groups of participants 
(here defined by gender). Nonetheless, these trends provide useful, preliminary information 
and suggest some interesting hypotheses for further research. 

EMPLOYMENT STATUS TRANSITIONS 

The quarterly employment rates present a static picture of employment by focusing on the 
"stock* of individuals who are employed during a given period of time- They fail to capture the 
dynamic aspect of employment, i.e v the "flow" of people into and out of employment. To 
illustrate this point, suppose a group of individuals had the same employment rate for two 
consecutive quarters. While this means that the same number of people were employed during 
both quarters, it does not mean that the same individuals were employed during both quarters. 
For example, it could be that the number of people who gained employment between the first 
and second quarters exactly offsets the number of people who lost employment during the same 
interval. Thus, observed stability in the quarterly employment rates could mask an active and 
shifting labor force. 

One way to explore labor force dynamics is to track the movement of individuals into and out 
of employment over the course of the post-program period. First, the employment status of 
each participant at termination is determined. Then, for each subsequent post-program quarter, 
the employment status of each participant is reexamined, and the participants are placed into 
groups defined on the basis of employment status during that quarter as well as previous 
employment history, This "branching" technique quickly expands the number of groups so that 
by the end of the fourth post-program quarter, 32 groups are identified, each possessing a 
different and unique set of post-program employment expc ienccs. For illustration, see Figure 
2 at the end of this chapter. 

When this technique was used to compare the post-program employment experiences of PY86 
and PY87 Title 1I-A adult terminees, the results for the two years showed a remarkable degree 
of similarity (see Figures 2 and 3), A comparison of the corresponding "transition rates" 14 reveals 
that the two sets of terminees were nearly identical in their movements into and out of 
employment during the post-program period. In fact, all of the transition rates for the PY87 
terminees were within two percentage points of their PY86 counterparts. 
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The data show a pronounced "momentum" for employment; see the transition rates along the 
top of Figures 2 and 3, which represent the quarter to quarter employment retention rates for 
those individuals who remained employed throughout the post-program period. For example, 
Figure 2 shows that the transition rate for this group increases from .79 (termination to first 
post-program quarter) to .90 (third to fourth post-program quarter). The increase in these rates 
over time confirms the intuition that the longer one remains employed, the less likely one is to 
become unemployed. 

Unfortunately, there appears to be a corresponding "stickiness" for unemployment. This is most 
apparent m the transition rates displayed along the bottom of Figures 2 and 3. These percentages 
get progressively larger over the course of the post-program period signifying that the longer a 
person remains unemployed, the less likely that person is to become employed in the future. 

It is also clear that whether a former participant is employed or not during a particular quarter 
is largely predicted by their employment status during the preceding quarter. Overall, 81% of 
the time, individuals remained in the same employment status from one quarter to the next. 

These results are further demonstrated in Figures 4 to 6 which combine the PY86 and PY87 
tcrmmees. The quarter-to-quarter transition rates for three subgroups of participants are 
depicted; females not receiving welfare (Figure 4), females receiving welfare (Figure 5), and 
males (Figure 6). The subgroups exhibit similar patterns of transition, although there are some 
minor variations in the levels of the rates. For example, although the momentum for 
employment is similar in both female subgroups, the employment-to-employment transition 
rates in the early post-program period are somewhat higher for females not receiving public 
assistance. In other words, females receiving public assistance at enrollment are more likely to 
become unemployed in the post-program period. 

One surprising finding was the relatively large percentage of participants (39% in PY86 and 40% 
in PY87) who were unemployed at termination according to JTPA MIS data but had UI wages 
in the first post-program quarter. This percentage is much larger than would be expected, based 
on the transition rates for later post-program periods, which were rarely above 30. 

To assess this finding, Ul-based employment status during the quarter of termination was used 
to calculate the transition rates instead of the MIS-based employment status at the date of 
termination. The previous comparison of UI and MIS data had revealed differences between 
the two data sources; the unexpectedly large percentage of jobgainerscould be the result of such 
differences. As expected, the transition rates based on the UI data (Figure 8) show a much lower 
proportion of job gainers" among U* initially unemployed group than indicated by the rates 
based on the MIS termination data (Figure 7): 27 versus .40, respectively. In addition, the 
proportion of "job losers" among the initially employed group using the UI data is lower than 
that using the MIS termination data: .15 versus .21, respectively. 

Further insight isgained by examining the correlationsamong SDA-lcvel employment measures 
calculated for different time points. For this analysis, employment rates based on MIS data 
(entered employment rate at termination) and UI data (employment rates for the quarter of 
termination and the post-program quarters) were calculated for each SDA. While there is a 
correlation of only .56 between the MIS-based initial employment rate and the first post-quarter 
employment rate, there isa much higher correlation of .94 between the Ul-based initial rate and 
the first post-quarter rate. In addition, the employment measure calculated from UI data for the 
quarter of termination is also strongly correlated with the longer-term employment measures. 
For example, the correlation between the Ul-based initial employment rate and the fourth 
post-quarter employment rate is .79. 
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The preceding exercise "illustrates the problems that can arise when mixing MIS and Ul data for 
purposes of tracking employment status over time. From the correlation between the MIS-based 
entered employment rate at termination and the Ul-based employment rate for the first 
post-quarter, an observer could erroneously conclude that there was only a mild relationship 
between employment outcomes for these two time points. However, the above evidence 
suggests that this interpretation is incorrect; the relatively low correlation is largely a function 
of using different data sources for calculating the two rates. Upon using the UI data throughout 
the analysis, a strong relationship is revealed. 

Summary And Conclusion 

This chapter has presented the findings from Phase II of NCEFs project to examine the potential 
for using UI wage-record data as an evaluation tool for programs funded under JTPA. Fifteen 
States participated in this phase, which covered Program Years 1986 and 1987. 

SYNOPSIS OF PHASE II FINDINGS 

The study began by assessing the telephone survey currently used to measure the employment 
and earnings experiences of JTPA participants during the 13th week after they leave the 
program. The analysis reinforced the findings of the earlier report from Phase I of the project: 
there appears to be substantial nonresponsebiasin the estimates of JTPA performance based on 
the post-program survey. Furthermore, DOL's current statistical adjustment procedure does 
not address all of the sources of bias found in the survey data. As a result, nonresponse bias in 
the survey-based measures of program outcomes can have a substantial effect on the 
interpretation of an SDA's performance. 

Efforts were also made to assess the degree of correspondence between the survey and UI data 
by using a subset of survey respondents whose 13-week post-program period coincided exactly 
with a calendar quarter. For the vast majority of these respondents, the two data sources were 
in agreement about the employment status of the respondent during the post-program period. 
However, for 20% of these respondents, the two data sources provided conflicting results. A 
detailed analysis explored the reasons for these conflicts, including out-of-state employment 
and the earn/ pay reporting lag. 

The third section of the chapter presented the pre/post employment and earnings trends for 
various subgroups of JTPA participants. For this analysis, the Phase 11 effort had expanded the 
Phase I database by bringing in four new States, covering PY87 as well as PY86 terminees, and 
including an additional year of post-program UI data for the PY86 terminees. The Phase II 
findings generally reinforced the Phase 1 findings. However, a few differences appeared; for 
example, the new results rejected an hypothesized delayed effect on employment among 
remedial /basic classroom training participants. 

The empirical analysis also documented the movement or "flow" of JTPA clients into and out of 
employment over the course of the post-program period. Among the findings, an individual's 
employment status during one quarter was found to be strongly correlated with their 
employment status in the previous quarter. In addition, Ul-based and MlS-based measures of 
initial employment rates were compared. 

IMPLICATIONS FOR JTPA PERFORMANCE MEASUREMENT 

The comparison of Ul-based and MIS-based initial employment rates demonstrated the 
problems that can arise in drawing inferences about employment transitions when different data 
sources arc used at different points in times. Fortunately, eight States provided UI information 




for the quarter of termination as well as the required pre- and post-program quarters. A 
Ul-based initial employment rate was constructed and compared to the MIS-based entered 
employment rate. The former proved more appropriate in tracking participants' employment 
transitions over time. 

The UI wage records offer a rich source of information on long-term employment patterns and 
trends. . . this study, JTPA participants were tracked for two years after they left the program, 
and even longer periods could be examined. The combination of these data with special survey 
data provide a powerful tool for program evaluation purposes. (See also the discussion in 
Chapter II.) 

Special surveys can be used by the States to collect employment information that cannot be 
gathered from UI records. For example, studies have indicated that employee benefits, 
particularly health insurance, are important determinants of job retention among economically 
disadvantaged workers. A survey can be used to gather this information from former 
participants in an effort to assess the quality of post-program employment. However, it is more 
important for States to collect detailed information on benefits offered by the job at placement. 
This information could then be used in conjunction with UI wage-record data to develop 
predictors of long-term employment retention. 

Special surveys can also be used to gather the opinions and attitudes of former clients about 
their experiences with JTPA. Information can be collected on why the client chose to enter JTPA, 
what factors influenced the services they received, and how they evaluate the quality of those 
services. The feedback from former participants can be helpful in uncovering program 
strengths and weaknesses. 

With regard to national performance standards, the recent elimination of immediate outcome 
measures, in favor of sole relianceonpost-programmeasures,hashad the undesirable side effect 
of reducing the ability of SDAs to manage their programs. When SDAs were assessed using 
termination-based measures, they could evaluate each service provider on the basis of their 
performance against the same yardstick. However, in the shift to post-program measures, many 
SDAs have lost this management tool because post-program data is collected on only a sample 
of their clients. The sample often contains too few cases on individual providers for SDAs to 
accurately assess their performance. 

The use of Ul-based performance measures would return this management tool to the SDAs. 
The costs of accessing UI data are so low that there would be no need to sample participants. 
As a result, SDAs would have access to post-program data for virtually all of their clients. This 
would place SDAs in the position where they could once again evaluate the effectiveness of their 
providers using performance measures set by the US. Department of Labor. 

The ability of the SDAs to manage their programs would be further enhanced by providing them 
with the technical assistance they need to evaluate their programs in terms of post-program 
outcomes. For example, national research efforts could be directed towards increasing the JTPA 
system's understanding of the factors that promote employment gains and losses in the 
post-program period. Are certain types of clients more likely to experience a job loss or a job 
gain after leaving the JTPA program? Are job losers and gainers more likely to have received 
certain types of training and employment services? To what extent are the observed transitions 
in labor force status a function of business cycles and other factors that are beyond the control 
of SDAs? Answers to such questions would help SDAs reestablish the link between program 
management and evaluation. 
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Figure 2 
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Figure 3 

Quarter to Quarter Employment Transition Rates 
PY87 Title n-A Adult Teiminees 
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Figure 4 

Quarter to Quarter Employment Transition Rates 
PY86 and PY87 Title II-A Adult Tenninees 
Females Not Receiving Welfare 
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Figure 5 

Quarter to Quarter Employment Transition Rates 
PY86 and PY87 Title H-A Adult Tenninees 
Females Receiving Welfare 
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Figure 6 

Quarter to Quarter Employment Transition Rates 
PY86 and FY87 Title II-A Adult Tenninees 
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Figure 7 

Quarter to Quarter Employment Transition Rates 
Employment Status at Termination Based on MIS Data* 
PY86 and PY87 Title II-A Adult Terminees 
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Figure 8 

Quarter to Quarter Employment Transition Rates 
Employment Status at Termination Based on UI Wage-Record Data* 
FY86 and PY87 Title II-A Adult Teiminees 
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Endnotes To Chapter IV 



1. The eleven States in Phase I were as follows: Florida, Idaho, Illinois, Indiana, Missouri, 
Nevada, Oregon, South Carolina, Utah, Virginia, and Washington. The database consisted of 
all JTPA participants who terminated from the program during Program Year 1966. Rrar 
quarters of pro-program and four quarters of post-program UI wages were requested for each 
participant 

2. Program Year 1986 is July 1, 1986 to June 30, 1987; and Program Year 1987 is July 1, 1987 to 
June 30, 1988. 7 

3. John Baj and Charles E. Trott, with David Stevens, A Feasibility Study of the Use of 
Unemploy ment Insurance Wage-Record Data as an Evaluation Tool for TTPA, Phase 1. National 
Commission for Employment Policy, Washington, DC, Research Report Number 90-02, 
January 1991. 

4. As noted above, nonrespondents to the survey could not be identified in two States. As a 
result, the response rates presented here are slightly inflated. 

5. Richard W. West, Katherine P. Dickinson and Catherine M. Casseriy, Performance 
Standards '89 : Managing Quality Programs, Follow-up Training Materials. Menlo Park,Ca: SRI 
International, 1989. 

6. The parameter estimates presented in Table 3 are somewhat different from those presented 
in the Phase I report. There are two reasons for these differences. First, the supplemental welfare 
sample was excluded in the Phase I analysis, but included in the Phase II analysis. Second, the 
Phase I analysis did not adjust for State differences in the overall response rate, i.e., dummy 
variables for States were not used. , 

7. Two of the Phase II States were excluded from this analysis because they did not provide 
information on nonrespondents, as discussed earlier. 

8. There is a fourth possible explanation for why a JTPA participant may not appear in UI 
wage-record data - the participant may have been employed by a "reimbursable employer." 
However, wage data are available for reimbursable employers in the State of Illinois (selected 
for special analysis below) and many other of the project States. 

9. Exempt status was inferred if the name of the employer was an individual rather man a 
company. 

10. The eight States for this analysis were: Florida, Idaho, Illinois, Kentucky, Maryland, 
Missouri, Nevada and Utah. Each had voluntarily provided UI data for the quarter of 
termination. 

11. The subgroups were defined on the basis of sex, race, educational status, welfare status 
and the type of training received while in the JTPA program. 

12. The study is being conducted by Dr. William Bowman. He is using a quasi-experimental 
design in the tradition of the work by Dr. James Heckman and his associates. For example, see 
Heckman, J. and V. J. Hotz, "Choosing Among Alternative Nonexperimental Methods for 
Estimating the Impact of Social Programs: The Case of Manpower Training," Journal of the 
American Statistical Association. Vol. 84, No. 408, December 1989, pp. 862-880. 
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13. A major study of the JTPA program is being conducted by Abt Associates for the U3. 
Department of Labor. The National JTPA Study is using a classical experimental design 
involving randomly selected control groups to measure the net impact of JTPA. Ifcrexamptes 
of State-level net impact efforts, see James Hanna and Zina Turney, The Net Iirawctof me 
Nevada JTPA Hue U Program," prepared for the Nevada State Job Training Office, 1988; and 
The Return on Investment From mdiana'sTiaining Programs Funded Through the Job Training 
Partnership Act," prejwed by the Research and Assessment Section of the Indiana Department 
of Employment and Training Services, 1986. 

14. The transition rafe between Status A and B is defined as the percentage of individuals in 
Status A at time T who were subsequently in Status Bat time T+l. 

15. The male population was not disaggregated into public assistance/ non-public assistance 
subgroups due to the relatively small number of males receiving public assistance. 

16. This analysis was based on information from the eight States that provided Ul wage-record 
data for the quarter for termination: Florida, Idaho, Illinois, Kentucky, Maryland, Missouri, 
Nevada and Utah. Since the transition rates produced from the data for th*^c States were nearly 
identical to those presented in Figure 3, they are not presented. 

17. Lois Thiessen Love, "rVnily Dependency and Job Retention: Implications" paper 
presented at the 1990 meeting of the American Evaluation Association, 1990. 
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Database Development 

Phase I Database 



Eleven States participated in Phase I of this project: Florida, Idaho, Illinois, Indiana, Missouri, 
Nevada, Oregon, South Carolina, Utah, Virginia, arv< Washington. They were required to 
submit two micro-data files to the clearinghouse. Ihe first data file contained demographic, 
program activity and program outcome information for each JTPA Title II-A and Title ITI adult 
and not-in-school youth participant who left the State's program during Program Year 1966 
(i J Y86). These data were extracted from the State's JTPA management information system 
(MIS) and included post-program data collected through the JTPA 1 3th-week follow-up survey. 

The second data file contained wage information extracted from the State's Unemployment 
Insurance (UI) wage records. For each person in the JTPA data file, me States were required to 
submit four quarters of pre-program UI wage information «md four quarters of post-program 
Ul wage information. The States located former JTPA clients on the UI database by using the 
clients' social security numbers as identifiers. When matches occurred, the wage data were 
extracted and stored in a separate rite. This UI extract file was then transferred to the 
clearinghouse along with the JTPA file. 

The JTPA and Ul extract files were merged by both the individual States and the clearinghouse, 
and each produced a series of data tables. The two sets of tables were compared in order to 
determine if there were any mis-communications regarding the coding of data items and to 
insure mat pre- and post-program quarters were being correctly identified. This data checking 
routine proved to be valuable tool for locating problems in the submitted data files. Further 
information regarding the procedures and techniques used to develop mis database can be 
found in the Phase I report. 

Phase n Database 



Phase II of this project was both an extension and expanston of the first phase. In Phase H, four 
new States were added to the list of already participating States - Georgia, Kentucky, Maryland, 
and Texas. Each of the new States was required to meet the data demands of Phase I as well as 
those of Phase II discussed below. In other words, these States were required to play n catch-up" 
by supplying the PY86 data previously submitted by the eleven continuing States. 

Phase II required each of the fifteen participating States to provide information on individuals 
who left the JTPA program during Program Year 1987 (PY87). 4 Similar to the request in Phase 
I, the States were asked to submit four quarters of pre-program Ul wage information and four 
quarters of post-program UI wage information for former JTPA clients. Although the States 
were not requested to submit information on in-school youth, a number of States chose to supply 
this information, as they had in Phase I. 

An important feature of Phase n was an extended tracking of the post-program experiences of 
PY86 program completers. Each State was asked to supply an additional four quarters of 
post-program UI wage data for this group of individuals. Access to this information allowed 
an extension of the gross outcomes analysis, developed in the Phase I study, to eight quarters 
of post-program experiences. These results are presented in die third section of this chapter. 
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The previous analysis of the Phase I data revealed a larger than expected decline in post-program 
employment rates between the third and fourth post-program quarters. In order to assess 
whether these declines were "real," or the result of incomplete fourth quarter data, the States 
were asked to reproduce these data in Phase II. If the decUnes were real me fourth quarter data 
submitted by the States for Phase II should have replicated the patterns found in Phase I. 
However, this did not occur. The fourth quarter data for PY86 program completers submitted 
in Phase II produced a third-to-four* quarter employment decline that was much less dramatic 
than the decline produced by the original Phase I data. This suggests that the fourth quarter 
data submitted in Phase I were incomplete. 

Two alternative explanations for the incomplete fourth quarter data were considered. One 
hypothesis was that the data were incomplete because of late reporting by employers to the 
State's UI system. In other words, the data had not been submitted to the clearinghouse because 
the information was not present in the State's UI system at the time the Phase I data were 
extracted. The other hypothesis was mat the information was in the State's UI systems but that 
problems occurred in extracting it Given the need to develop matching algorithms to access UI 
information for JTPA program completers, undetected data-access problems were quite 
possible. 

A comparison of the original fourth quarter data submitted in Phase I with the corresponding 
data submitted in Phase II revealed mat data extract errors were the major explanation. In 
several States, there were numerous instances in which fourth quarter data submitted in Phase 
1 were not found in the Phase II reproduction of this quarter. Efforts by State staff to track down 
the reason for the disappearance of these data invariably revealed a data extract problem. Once 
these problems were corrected, new files were generated for both the PY86 and PY87 program 
completers. 

The presence of data extract problems in Phase I does not eliminate the possibility that late 
reporting also played a role in the observed third-to-fourth quarter decline in employment rates. 
To assess this possibility, the post-program patterns produced by the UI data submitted for PY87 
program completers were examined. If late reporting by employers was a major source of the 
relatively large third-to-fourth quarter decline in employment found in the original Phase I data, 
one would expect to see an equally large decline between these two quarters in the Phase II data 
for PY87 program completers. However, if this large decline did not occur, the inference from 
this evidence would be that data extract errors were the crucial issue. 

This analysis supported the conclusion that data extract errors were the primary cause of the 
large third-to-fourth quarter employment declines found in the original Phase I data (see Figure 
1). The post-program employment partem found for FY87 program completers looks very 
similar to the pattern exhibited for PY86 program completers using the corrected Phase II data. 
Since late reporting by employers to the UI system is not a factor in the corrected fourth quarter 
data for PY86 program completers, the similarity of these patterns suggest that late reporting 
by employers is a minor concern. 5 This conclusion is further supported by the gradual decline 
exhibited between the seventh and eight post-program quarters for PY86 program completers. 
If late reporting by employers was a major problem, one would expect a more dramatic decline 
in employment between these two quarters. 

There were also differences between the Phase I and Phase 11 databases with regard to the 
post-program survey data. Some of the States participating in Phase I rebuilt their JTPA data 
files and resubmitted them to the clearinghouse during Phase II. In the process of reconstructing 
the Phase 1 database, it became apparent that some of the revised data files contained more 
post-program survey data than the previously submitted files, while others contained less 
survey data. The States were asked to explain such discrepancies between the two data 
submissions. The usual explanation was a computer programming problem in merging the 
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Figure 1 

Comparison of Phase I and Phase II Data 
Post-Program Employment Trends 
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post-program survey data with the other JTPA data. 6 The end result was that there is slightly 
more post-program survey data for PY86 program completers in the current database. 

The construction of the Phase II database was delayed due to the inability of two States to supply 
the clearinghouse with data files within the time frame allowed for this activity. In one of the 
States, the problem resulted from staff turnover. The individual who had processed the data 
for Phase I left the State agency without documenting the procedures used to extract the data. 
The new person in this position had to recreate a number of computer programs, forcing a delay 
in production of the data. 

In the second State, the problem stemmed from a temporary reduction in the State' s computer 
resources and support This State requested and received an extension of the deadline for 
producing the Phase II data. Although this resource limitation was eventually resolved, it 
caused a substantial delay in the creation of the Phase II database. 

Both of the States that experienced difficulties in producing the Phase II data informed the 
clearinghouse that their problems were temporary. Both of these States had participated in 
Phase I of mis project and neither had experienced any problems in meeting the deadlines set 
in the earlier phase. 

There are some gaps in the project's database Most of these gaps were identified and disc u ssed 
in the Phase I report cited above. The remainder of this appendix focuses attention on new data 
problems which emerged in Phase II. 

Three of the four new States could not supply all of the requested data. One State had relied on 
a previously constructed Ul extract file to meet the data needs of the current project. By relying 
on this extract file, the State could not provide all of the requested pro-program UI information. 
In addition, the State also reported that post-program UI data for Title III participants were not 
available prior to the first quarter of PY87. However, complete post-program UI data were 
available for all of the State's Title II- A participants. 

The other two States were unable to identify individuals in the 1 3th- week follow-up sample who 
had not responded to the telephone survey. Although the data gathered from former JTPA 
clients who responded to the survey weie stored on their JTPA data system, it was impossible 
to identify individuals who were selected for the survey but did not respond. As a result the 
data from these States could not be used in analyses requiring knowledge of both respondents 
and nonrespondents to the post-program survey. 

Finally, one of the latter two States did not supply complete post-program UI data for its PY86 
program completers. This State tad unusually low Ul-based employment rates for the fifth to 
eight post-program quarters. Further analysis revealed a computer programming problem in 
extracting the UI information. Rather than delay the production of this chapter, the decision 
was made to exclude this State's data from analyses requiring complete post-program 
information. 
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Endnotes To Appendix IV.A 



1. For technical reasons, the clearinghouse requested that the States submit the information 
described for the JTPA data file in two separate data sets. To simplify the present discussion, 
these two data sets are characterized as a single file. 

2. Program Year 1986 is July 1, 1986 to June 30, 1987. 

3. John Baj and Charles E Trott, with David Stevens, A Feasibility Study of the Use of 
Unemployment Insurance Wage-Record Data as an Evaluation Tool for TTPA, Phase I. National 
Commission for Employment Policy, Washington, D.C, Research Report Number 904)2, 
January 1991. 

4. Program Year 1987 is July 1, 1987 to June 30, 1988. 

S^The assumption that late reporting is rare in the corrected PY86 data is based on the 
observation that employers would have had over one year to submit their wage information by 
the time tl e UI files were accessed the second time. 

6. Many States employ outside contractors to collect the post-program survey data. Most of 
these contractors do not have on-line access to the JTPA MB. As a result, the data they collect 
are not directed entered into the MIS system but instead are merged with the larger JTPA 
database through the use of separate computer programs. 
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APPENDIX IV.B 

Employment And Earnings Trends 



Program Years 1986 and 1987 
JTP \ Title II-A Adult Terminees 

The following figures display the pre-program and post-program employment and earnings 
trends for adult terminees from JTPA's Title II-A program. The section labeled "PROGRAM" in 
the middle of the figures refers to the in-program period, including the quarter of enrollment 
and the quarter of termination. 

For Figures 1 - 5, the quarterly employment rates are defined as the percentage of terminees who 
hadUl wages reported for a given quarter. The earnings displayed in Figures 6 -10 are defined 
as the average earnings of employed workers. 
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Ffgura B,1a 
Employment Trends By Sax 
Program Ywr 1986 
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Figure B.1b 
Employment Trencfa By Sex 
Program Year 1987 
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Figure B.2s 
Employment Trantfs By RacaTEthnlclty 
Program Y#ar 1386 
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Figure B.3a 
Employment Trends By Educational Status 
Program Year 1986 
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Figure B.3b 
Employment Trends By Educational Status 
Program Year 1987 
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Figure B.4a 
Employment Trends By Waif are Status 
Program Year 19B6 
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Figure B.5a 
Employment Trends By Program Activity 
Program Yaar 1986 
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Figure B.6b 
Earnings Trends By Sex 
Program Year 1987 
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Rgura B.7a 
Earning! Trends By Raca/Ethnldty 
Program Vaar 1986 
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Flgura B.7b 
Earnings Trend* By Race/Ethnicity 
Program Yaar 1987 
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Figuf* B.8a 
Earnings Trands By Educational Status 
Program Yaar 1986 
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Flguro B.8b 
Earnings Trends By Educational Status 
Program Year 1987 
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Figure B.9a 
Earnings Trends By Welfare Status 
Program Year 1986 
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Figure B.9b 
Earnings Trend* By Welfare Status 
Program Year 1987 
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Figure B,10a 
Earnings Trends By Program Activity 
Program Yaar 1986 




Figure B.10b 
Earnings Trends By Program Activity 
Program Year 1987 
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